Fold 0
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': None,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_0',
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': None,
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.268350
Best score so far is 0.268350 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.521759
Epoch 2
Validation binary_cross_entropy = 0.325468
Epoch 3
Validation binary_cross_entropy = 0.458015
Epoch 4
Loss = 1.1453e-02, PNorm = 87.7210, GNorm = 4.6520, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.311948
Epoch 5
Validation binary_cross_entropy = 0.691214
Epoch 6
Validation binary_cross_entropy = 0.292078
Epoch 7
Validation binary_cross_entropy = 0.473392
Epoch 8
Validation binary_cross_entropy = 0.463401
Epoch 9
Loss = 6.4295e-03, PNorm = 88.0208, GNorm = 1.7360, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.315265
Epoch 10
Validation binary_cross_entropy = 0.532591
Epoch 11
Validation binary_cross_entropy = 0.414028
Epoch 12
Validation binary_cross_entropy = 0.344516
Epoch 13
Validation binary_cross_entropy = 0.553586
Epoch 14
Loss = 5.0271e-03, PNorm = 88.3444, GNorm = 1.5204, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.395493
Epoch 15
Validation binary_cross_entropy = 0.331212
Epoch 16
Validation binary_cross_entropy = 0.448045
Epoch 17
Validation binary_cross_entropy = 0.431906
Epoch 18
Validation binary_cross_entropy = 0.365776
Epoch 19
Loss = 3.7839e-03, PNorm = 88.5922, GNorm = 1.9521, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.403697
Epoch 20
Validation binary_cross_entropy = 0.455268
Epoch 21
Validation binary_cross_entropy = 0.358930
Epoch 22
Validation binary_cross_entropy = 0.350240
Epoch 23
Validation binary_cross_entropy = 0.452292
Epoch 24
Loss = 4.1726e-03, PNorm = 88.7835, GNorm = 2.2083, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.482963
Epoch 25
Validation binary_cross_entropy = 0.388247
Epoch 26
Validation binary_cross_entropy = 0.354753
Epoch 27
Validation binary_cross_entropy = 0.384450
Epoch 28
Validation binary_cross_entropy = 0.334835
Epoch 29
Loss = 4.0330e-03, PNorm = 88.9650, GNorm = 3.5640, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.349973
Epoch 30
Validation binary_cross_entropy = 0.434160
Epoch 31
Validation binary_cross_entropy = 0.401314
Epoch 32
Validation binary_cross_entropy = 0.343832
Epoch 33
Validation binary_cross_entropy = 0.374548
Epoch 34
Loss = 2.3795e-03, PNorm = 89.1521, GNorm = 2.7874, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.384351
Epoch 35
Validation binary_cross_entropy = 0.371872
Epoch 36
Validation binary_cross_entropy = 0.379231
Epoch 37
Validation binary_cross_entropy = 0.373250
Epoch 38
Validation binary_cross_entropy = 0.369438
Epoch 39
Loss = 1.4364e-03, PNorm = 89.3339, GNorm = 0.7269, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.366894
Epoch 40
Validation binary_cross_entropy = 0.382459
Epoch 41
Validation binary_cross_entropy = 0.402448
Epoch 42
Validation binary_cross_entropy = 0.443954
Epoch 43
Validation binary_cross_entropy = 0.415520
Epoch 44
Loss = 1.0815e-03, PNorm = 89.4789, GNorm = 0.9043, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.416553
Epoch 45
Validation binary_cross_entropy = 0.399885
Epoch 46
Validation binary_cross_entropy = 0.409266
Epoch 47
Validation binary_cross_entropy = 0.433834
Epoch 48
Validation binary_cross_entropy = 0.475326
Epoch 49
Loss = 1.9276e-03, PNorm = 89.6226, GNorm = 0.9466, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.506286
Epoch 50
Validation binary_cross_entropy = 0.506725
Epoch 51
Validation binary_cross_entropy = 0.499806
Epoch 52
Validation binary_cross_entropy = 0.486586
Epoch 53
Validation binary_cross_entropy = 0.557844
Epoch 54
Loss = 2.6860e-03, PNorm = 89.7760, GNorm = 1.9630, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.471375
Epoch 55
Validation binary_cross_entropy = 0.494877
Epoch 56
Validation binary_cross_entropy = 0.454802
Epoch 57
Validation binary_cross_entropy = 0.466135
Epoch 58
Validation binary_cross_entropy = 0.494856
Epoch 59
Loss = 1.3110e-03, PNorm = 89.9367, GNorm = 2.0939, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.476008
Epoch 60
Validation binary_cross_entropy = 0.445199
Epoch 61
Validation binary_cross_entropy = 0.483130
Epoch 62
Validation binary_cross_entropy = 0.449615
Epoch 63
Validation binary_cross_entropy = 0.484302
Epoch 64
Loss = 1.6839e-03, PNorm = 90.0996, GNorm = 2.3782, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.465987
Epoch 65
Validation binary_cross_entropy = 0.461405
Epoch 66
Validation binary_cross_entropy = 0.470932
Epoch 67
Validation binary_cross_entropy = 0.481451
Epoch 68
Validation binary_cross_entropy = 0.449707
Epoch 69
Loss = 9.5981e-04, PNorm = 90.2903, GNorm = 0.3453, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.442174
Epoch 70
Validation binary_cross_entropy = 0.472884
Epoch 71
Validation binary_cross_entropy = 0.501603
Epoch 72
Validation binary_cross_entropy = 0.474156
Epoch 73
Validation binary_cross_entropy = 0.476749
Epoch 74
Loss = 7.6271e-04, PNorm = 90.4937, GNorm = 0.5411, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.489835
Epoch 75
Validation binary_cross_entropy = 0.484491
Epoch 76
Validation binary_cross_entropy = 0.566239
Epoch 77
Validation binary_cross_entropy = 0.553458
Epoch 78
Validation binary_cross_entropy = 0.486420
Epoch 79
Loss = 2.9281e-03, PNorm = 90.6785, GNorm = 1.9912, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.486041
Epoch 80
Validation binary_cross_entropy = 0.466446
Epoch 81
Validation binary_cross_entropy = 0.471640
Epoch 82
Validation binary_cross_entropy = 0.465084
Epoch 83
Validation binary_cross_entropy = 0.468910
Epoch 84
Loss = 1.6212e-03, PNorm = 90.8899, GNorm = 1.4498, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.521003
Epoch 85
Validation binary_cross_entropy = 0.524822
Epoch 86
Validation binary_cross_entropy = 0.495284
Epoch 87
Validation binary_cross_entropy = 0.489337
Epoch 88
Validation binary_cross_entropy = 0.481255
Epoch 89
Loss = 1.8935e-03, PNorm = 91.0964, GNorm = 0.8925, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.529919
Epoch 90
Validation binary_cross_entropy = 0.480343
Epoch 91
Validation binary_cross_entropy = 0.422812
Epoch 92
Validation binary_cross_entropy = 0.427133
Epoch 93
Validation binary_cross_entropy = 0.420403
Epoch 94
Loss = 1.0495e-03, PNorm = 91.2954, GNorm = 0.4441, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.421239
Epoch 95
Validation binary_cross_entropy = 0.463504
Epoch 96
Validation binary_cross_entropy = 0.463418
Epoch 97
Validation binary_cross_entropy = 0.457329
Epoch 98
Validation binary_cross_entropy = 0.486619
Epoch 99
Loss = 3.4705e-03, PNorm = 91.5023, GNorm = 1.4507, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.525339
Epoch 100
Validation binary_cross_entropy = 0.625300
Epoch 101
Validation binary_cross_entropy = 0.564992
Epoch 102
Validation binary_cross_entropy = 0.497335
Epoch 103
Validation binary_cross_entropy = 0.486351
Epoch 104
Loss = 1.0341e-03, PNorm = 91.7383, GNorm = 1.2946, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.471587
Epoch 105
Validation binary_cross_entropy = 0.448770
Epoch 106
Validation binary_cross_entropy = 0.448311
Epoch 107
Validation binary_cross_entropy = 0.451767
Epoch 108
Validation binary_cross_entropy = 0.466135
Epoch 109
Loss = 1.8905e-03, PNorm = 91.9140, GNorm = 0.3119, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.552051
Epoch 110
Validation binary_cross_entropy = 0.645721
Epoch 111
Validation binary_cross_entropy = 0.644914
Epoch 112
Validation binary_cross_entropy = 0.539977
Epoch 113
Validation binary_cross_entropy = 0.487199
Epoch 114
Loss = 6.7683e-04, PNorm = 92.0841, GNorm = 0.8758, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.481816
Epoch 115
Validation binary_cross_entropy = 0.486928
Epoch 116
Validation binary_cross_entropy = 0.512334
Epoch 117
Validation binary_cross_entropy = 0.567153
Epoch 118
Validation binary_cross_entropy = 0.556535
Epoch 119
Loss = 5.4539e-04, PNorm = 92.2406, GNorm = 0.5573, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.538252
Epoch 120
Validation binary_cross_entropy = 0.519398
Epoch 121
Validation binary_cross_entropy = 0.517055
Epoch 122
Validation binary_cross_entropy = 0.525928
Epoch 123
Validation binary_cross_entropy = 0.537664
Epoch 124
Loss = 7.2123e-04, PNorm = 92.3886, GNorm = 0.2443, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.563952
Epoch 125
Validation binary_cross_entropy = 0.614338
Epoch 126
Validation binary_cross_entropy = 0.607052
Epoch 127
Validation binary_cross_entropy = 0.574456
Epoch 128
Validation binary_cross_entropy = 0.560683
Epoch 129
Loss = 3.2994e-04, PNorm = 92.5193, GNorm = 0.0347, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.593750
Epoch 130
Validation binary_cross_entropy = 0.638232
Epoch 131
Validation binary_cross_entropy = 0.662667
Epoch 132
Validation binary_cross_entropy = 0.583782
Epoch 133
Validation binary_cross_entropy = 0.564428
Epoch 134
Loss = 4.1388e-04, PNorm = 92.6152, GNorm = 0.9339, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.553551
Epoch 135
Validation binary_cross_entropy = 0.553783
Epoch 136
Validation binary_cross_entropy = 0.559788
Epoch 137
Validation binary_cross_entropy = 0.558899
Epoch 138
Validation binary_cross_entropy = 0.580042
Epoch 139
Loss = 1.3582e-04, PNorm = 92.7586, GNorm = 0.1905, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.600044
Epoch 140
Validation binary_cross_entropy = 0.622730
Epoch 141
Validation binary_cross_entropy = 0.634632
Epoch 142
Validation binary_cross_entropy = 0.607973
Epoch 143
Validation binary_cross_entropy = 0.594510
Epoch 144
Loss = 2.4425e-04, PNorm = 92.8929, GNorm = 0.7603, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.591435
Epoch 145
Validation binary_cross_entropy = 0.605186
Epoch 146
Validation binary_cross_entropy = 0.628220
Epoch 147
Validation binary_cross_entropy = 0.645077
Epoch 148
Validation binary_cross_entropy = 0.625995
Epoch 149
Loss = 5.7164e-05, PNorm = 93.0347, GNorm = 0.0580, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.613561
Epoch 150
Validation binary_cross_entropy = 0.613180
Epoch 151
Validation binary_cross_entropy = 0.615269
Epoch 152
Validation binary_cross_entropy = 0.630293
Epoch 153
Validation binary_cross_entropy = 0.670306
Epoch 154
Loss = 4.7325e-04, PNorm = 93.1307, GNorm = 1.4734, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.690023
Epoch 155
Validation binary_cross_entropy = 0.690298
Epoch 156
Validation binary_cross_entropy = 0.696451
Epoch 157
Validation binary_cross_entropy = 0.699093
Epoch 158
Validation binary_cross_entropy = 0.707392
Epoch 159
Loss = 6.1966e-05, PNorm = 93.2054, GNorm = 0.1004, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.726242
Epoch 160
Validation binary_cross_entropy = 0.693919
Epoch 161
Validation binary_cross_entropy = 0.647734
Epoch 162
Validation binary_cross_entropy = 0.642527
Epoch 163
Validation binary_cross_entropy = 0.679693
Epoch 164
Loss = 4.5662e-04, PNorm = 93.3165, GNorm = 1.2840, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.712614
Epoch 165
Validation binary_cross_entropy = 0.665972
Epoch 166
Validation binary_cross_entropy = 0.650285
Epoch 167
Validation binary_cross_entropy = 0.651679
Epoch 168
Validation binary_cross_entropy = 0.660311
Epoch 169
Loss = 6.4065e-05, PNorm = 93.4173, GNorm = 0.0133, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.668440
Epoch 170
Validation binary_cross_entropy = 0.674988
Epoch 171
Validation binary_cross_entropy = 0.687990
Epoch 172
Validation binary_cross_entropy = 0.704277
Epoch 173
Validation binary_cross_entropy = 0.704920
Epoch 174
Loss = 4.0924e-04, PNorm = 93.5012, GNorm = 0.4575, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.709837
Epoch 175
Validation binary_cross_entropy = 0.661197
Epoch 176
Validation binary_cross_entropy = 0.639633
Epoch 177
Validation binary_cross_entropy = 0.655775
Epoch 178
Validation binary_cross_entropy = 0.689984
Epoch 179
Loss = 2.3937e-03, PNorm = 93.5827, GNorm = 1.5790, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.690708
Epoch 180
Validation binary_cross_entropy = 0.774324
Epoch 181
Validation binary_cross_entropy = 0.806170
Epoch 182
Validation binary_cross_entropy = 0.751989
Epoch 183
Validation binary_cross_entropy = 0.703695
Epoch 184
Loss = 3.0244e-04, PNorm = 93.6815, GNorm = 0.8959, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.682800
Epoch 185
Validation binary_cross_entropy = 0.703334
Epoch 186
Validation binary_cross_entropy = 0.704620
Epoch 187
Validation binary_cross_entropy = 0.687212
Epoch 188
Validation binary_cross_entropy = 0.699625
Epoch 189
Loss = 1.0562e-03, PNorm = 93.7720, GNorm = 2.0195, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.687005
Epoch 190
Validation binary_cross_entropy = 0.681630
Epoch 191
Validation binary_cross_entropy = 0.701225
Epoch 192
Validation binary_cross_entropy = 0.708054
Epoch 193
Validation binary_cross_entropy = 0.710962
Epoch 194
Loss = 1.5915e-04, PNorm = 93.8546, GNorm = 0.7277, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.722036
Epoch 195
Validation binary_cross_entropy = 0.724554
Epoch 196
Validation binary_cross_entropy = 0.741154
Epoch 197
Validation binary_cross_entropy = 0.802191
Epoch 198
Validation binary_cross_entropy = 0.785121
Epoch 199
Loss = 6.7068e-04, PNorm = 93.9504, GNorm = 1.1779, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.765983
Epoch 200
Validation binary_cross_entropy = 0.762178
Epoch 201
Validation binary_cross_entropy = 0.757204
Epoch 202
Validation binary_cross_entropy = 0.748123
Epoch 203
Validation binary_cross_entropy = 0.753525
Epoch 204
Loss = 1.4081e-04, PNorm = 94.0412, GNorm = 0.2369, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.762466
Epoch 205
Validation binary_cross_entropy = 0.746936
Epoch 206
Validation binary_cross_entropy = 0.758159
Epoch 207
Validation binary_cross_entropy = 0.762923
Epoch 208
Validation binary_cross_entropy = 0.716410
Epoch 209
Loss = 7.6215e-05, PNorm = 94.1166, GNorm = 0.2815, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.692948
Epoch 210
Validation binary_cross_entropy = 0.702105
Epoch 211
Validation binary_cross_entropy = 0.730954
Epoch 212
Validation binary_cross_entropy = 0.764534
Epoch 213
Validation binary_cross_entropy = 0.791416
Epoch 214
Loss = 2.6422e-04, PNorm = 94.2212, GNorm = 0.9022, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.751349
Epoch 215
Validation binary_cross_entropy = 0.730006
Epoch 216
Validation binary_cross_entropy = 0.730413
Epoch 217
Validation binary_cross_entropy = 0.750071
Epoch 218
Validation binary_cross_entropy = 0.822219
Epoch 219
Loss = 5.7781e-04, PNorm = 94.3141, GNorm = 0.3593, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.839924
Epoch 220
Validation binary_cross_entropy = 0.811069
Epoch 221
Validation binary_cross_entropy = 0.793594
Epoch 222
Validation binary_cross_entropy = 0.783731
Epoch 223
Validation binary_cross_entropy = 0.780126
Epoch 224
Loss = 1.6106e-03, PNorm = 94.4117, GNorm = 1.5897, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.811566
Epoch 225
Validation binary_cross_entropy = 0.893449
Epoch 226
Validation binary_cross_entropy = 0.630019
Epoch 227
Validation binary_cross_entropy = 0.516042
Epoch 228
Validation binary_cross_entropy = 0.491095
Epoch 229
Loss = 3.3565e-04, PNorm = 94.5934, GNorm = 0.6651, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.497846
Epoch 230
Validation binary_cross_entropy = 0.615474
Epoch 231
Validation binary_cross_entropy = 0.687109
Epoch 232
Validation binary_cross_entropy = 0.677165
Epoch 233
Validation binary_cross_entropy = 0.748526
Epoch 234
Loss = 1.2349e-04, PNorm = 94.8524, GNorm = 0.2300, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.773903
Epoch 235
Validation binary_cross_entropy = 0.732582
Epoch 236
Validation binary_cross_entropy = 0.729093
Epoch 237
Validation binary_cross_entropy = 0.739162
Epoch 238
Validation binary_cross_entropy = 0.753621
Epoch 239
Loss = 5.4638e-04, PNorm = 95.0927, GNorm = 1.7688, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.818114
Epoch 240
Validation binary_cross_entropy = 0.866786
Epoch 241
Validation binary_cross_entropy = 0.864217
Epoch 242
Validation binary_cross_entropy = 0.789986
Epoch 243
Validation binary_cross_entropy = 0.755838
Epoch 244
Loss = 5.3520e-05, PNorm = 95.2924, GNorm = 0.1140, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.759144
Epoch 245
Validation binary_cross_entropy = 0.762180
Epoch 246
Validation binary_cross_entropy = 0.757681
Epoch 247
Validation binary_cross_entropy = 0.774926
Epoch 248
Validation binary_cross_entropy = 0.805712
Epoch 249
Loss = 1.0383e-04, PNorm = 95.4401, GNorm = 0.2767, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.842835
Epoch 250
Validation binary_cross_entropy = 0.855721
Epoch 251
Validation binary_cross_entropy = 0.825612
Epoch 252
Validation binary_cross_entropy = 0.787602
Epoch 253
Validation binary_cross_entropy = 0.752996
Epoch 254
Loss = 2.8360e-05, PNorm = 95.5440, GNorm = 0.0734, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.729504
Epoch 255
Validation binary_cross_entropy = 0.715287
Epoch 256
Validation binary_cross_entropy = 0.714111
Epoch 257
Validation binary_cross_entropy = 0.721315
Epoch 258
Validation binary_cross_entropy = 0.733510
Epoch 259
Loss = 4.9723e-05, PNorm = 95.6091, GNorm = 0.2999, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.750067
Epoch 260
Validation binary_cross_entropy = 0.773721
Epoch 261
Validation binary_cross_entropy = 0.800663
Epoch 262
Validation binary_cross_entropy = 0.823623
Epoch 263
Validation binary_cross_entropy = 0.833879
Epoch 264
Loss = 1.4843e-04, PNorm = 95.6517, GNorm = 0.6770, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.823891
Epoch 265
Validation binary_cross_entropy = 0.803525
Epoch 266
Validation binary_cross_entropy = 0.807402
Epoch 267
Validation binary_cross_entropy = 0.853153
Epoch 268
Validation binary_cross_entropy = 0.917013
Epoch 269
Loss = 1.3384e-04, PNorm = 95.6971, GNorm = 0.4541, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.939993
Epoch 270
Validation binary_cross_entropy = 0.797837
Epoch 271
Validation binary_cross_entropy = 0.714900
Epoch 272
Validation binary_cross_entropy = 0.706374
Epoch 273
Validation binary_cross_entropy = 0.693383
Epoch 274
Loss = 9.7206e-05, PNorm = 95.7903, GNorm = 0.7824, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.731624
Epoch 275
Validation binary_cross_entropy = 0.798357
Epoch 276
Validation binary_cross_entropy = 0.822913
Epoch 277
Validation binary_cross_entropy = 0.811001
Epoch 278
Validation binary_cross_entropy = 0.806224
Epoch 279
Loss = 9.5342e-04, PNorm = 95.9172, GNorm = 2.5584, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.717766
Epoch 280
Validation binary_cross_entropy = 0.667620
Epoch 281
Validation binary_cross_entropy = 0.666965
Epoch 282
Validation binary_cross_entropy = 0.659891
Epoch 283
Validation binary_cross_entropy = 0.648464
Epoch 284
Loss = 6.4678e-05, PNorm = 96.0132, GNorm = 0.1948, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.693348
Epoch 285
Validation binary_cross_entropy = 0.755659
Epoch 286
Validation binary_cross_entropy = 0.728767
Epoch 287
Validation binary_cross_entropy = 0.645360
Epoch 288
Validation binary_cross_entropy = 0.585677
Epoch 289
Loss = 7.7727e-05, PNorm = 96.1642, GNorm = 0.2166, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.577327
Epoch 290
Validation binary_cross_entropy = 0.594725
Epoch 291
Validation binary_cross_entropy = 0.597748
Epoch 292
Validation binary_cross_entropy = 0.628665
Epoch 293
Validation binary_cross_entropy = 0.710777
Epoch 294
Loss = 4.1686e-05, PNorm = 96.2776, GNorm = 0.1597, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.790292
Epoch 295
Validation binary_cross_entropy = 0.857411
Epoch 296
Validation binary_cross_entropy = 0.892927
Epoch 297
Validation binary_cross_entropy = 0.853737
Epoch 298
Validation binary_cross_entropy = 0.801570
Epoch 299
Loss = 1.8866e-04, PNorm = 96.3541, GNorm = 0.3960, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.758423
Model 0 best validation binary_cross_entropy = 0.268350 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.220527
Ensemble test binary_cross_entropy = 0.220527
Fold 1
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_1',
 'save_smiles_splits': False,
 'seed': 1,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.277389
Best score so far is 0.277389 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.448765
Epoch 2
Validation binary_cross_entropy = 0.324402
Epoch 3
Validation binary_cross_entropy = 0.674570
Epoch 4
Loss = 8.6760e-03, PNorm = 87.7257, GNorm = 1.9488, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.308040
Epoch 5
Validation binary_cross_entropy = 0.711298
Epoch 6
Validation binary_cross_entropy = 0.328983
Epoch 7
Validation binary_cross_entropy = 0.396677
Epoch 8
Validation binary_cross_entropy = 0.414767
Epoch 9
Loss = 6.5820e-03, PNorm = 88.0350, GNorm = 1.9769, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.328942
Epoch 10
Validation binary_cross_entropy = 0.845570
Epoch 11
Validation binary_cross_entropy = 0.320166
Epoch 12
Validation binary_cross_entropy = 0.375695
Epoch 13
Validation binary_cross_entropy = 0.488079
Epoch 14
Loss = 6.8954e-03, PNorm = 88.3589, GNorm = 1.9340, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.378784
Epoch 15
Validation binary_cross_entropy = 0.396068
Epoch 16
Validation binary_cross_entropy = 0.374682
Epoch 17
Validation binary_cross_entropy = 0.304386
Epoch 18
Validation binary_cross_entropy = 0.300872
Epoch 19
Loss = 4.1397e-03, PNorm = 88.6057, GNorm = 1.8323, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.488425
Epoch 20
Validation binary_cross_entropy = 0.483801
Epoch 21
Validation binary_cross_entropy = 0.361251
Epoch 22
Validation binary_cross_entropy = 0.389029
Epoch 23
Validation binary_cross_entropy = 0.480881
Epoch 24
Loss = 4.2356e-03, PNorm = 88.7991, GNorm = 2.2765, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.385362
Epoch 25
Validation binary_cross_entropy = 0.351171
Epoch 26
Validation binary_cross_entropy = 0.354076
Epoch 27
Validation binary_cross_entropy = 0.428599
Epoch 28
Validation binary_cross_entropy = 0.390828
Epoch 29
Loss = 3.4856e-03, PNorm = 88.9791, GNorm = 1.0676, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.352529
Epoch 30
Validation binary_cross_entropy = 0.373173
Epoch 31
Validation binary_cross_entropy = 0.424183
Epoch 32
Validation binary_cross_entropy = 0.406235
Epoch 33
Validation binary_cross_entropy = 0.406051
Epoch 34
Loss = 2.4590e-03, PNorm = 89.1421, GNorm = 2.1850, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.405725
Epoch 35
Validation binary_cross_entropy = 0.413563
Epoch 36
Validation binary_cross_entropy = 0.440161
Epoch 37
Validation binary_cross_entropy = 0.446918
Epoch 38
Validation binary_cross_entropy = 0.439023
Epoch 39
Loss = 4.6406e-03, PNorm = 89.2956, GNorm = 2.6808, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.425568
Epoch 40
Validation binary_cross_entropy = 0.428080
Epoch 41
Validation binary_cross_entropy = 0.420525
Epoch 42
Validation binary_cross_entropy = 0.421707
Epoch 43
Validation binary_cross_entropy = 0.412234
Epoch 44
Loss = 3.2397e-03, PNorm = 89.4463, GNorm = 1.6277, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.456306
Epoch 45
Validation binary_cross_entropy = 0.461713
Epoch 46
Validation binary_cross_entropy = 0.479013
Epoch 47
Validation binary_cross_entropy = 0.499022
Epoch 48
Validation binary_cross_entropy = 0.458550
Epoch 49
Loss = 2.9474e-03, PNorm = 89.6707, GNorm = 1.0930, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.435951
Epoch 50
Validation binary_cross_entropy = 0.411930
Epoch 51
Validation binary_cross_entropy = 0.434026
Epoch 52
Validation binary_cross_entropy = 0.403309
Epoch 53
Validation binary_cross_entropy = 0.444528
Epoch 54
Loss = 3.1124e-03, PNorm = 89.8749, GNorm = 2.4601, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.428481
Epoch 55
Validation binary_cross_entropy = 0.427306
Epoch 56
Validation binary_cross_entropy = 0.459484
Epoch 57
Validation binary_cross_entropy = 0.421189
Epoch 58
Validation binary_cross_entropy = 0.513664
Epoch 59
Loss = 1.9242e-03, PNorm = 90.0571, GNorm = 2.2636, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.518495
Epoch 60
Validation binary_cross_entropy = 0.418489
Epoch 61
Validation binary_cross_entropy = 0.436498
Epoch 62
Validation binary_cross_entropy = 0.445986
Epoch 63
Validation binary_cross_entropy = 0.454090
Epoch 64
Loss = 1.9324e-03, PNorm = 90.2335, GNorm = 1.3493, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.444677
Epoch 65
Validation binary_cross_entropy = 0.460001
Epoch 66
Validation binary_cross_entropy = 0.457753
Epoch 67
Validation binary_cross_entropy = 0.436009
Epoch 68
Validation binary_cross_entropy = 0.441542
Epoch 69
Loss = 1.0413e-03, PNorm = 90.3743, GNorm = 0.7914, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.463602
Epoch 70
Validation binary_cross_entropy = 0.496994
Epoch 71
Validation binary_cross_entropy = 0.509031
Epoch 72
Validation binary_cross_entropy = 0.500937
Epoch 73
Validation binary_cross_entropy = 0.470041
Epoch 74
Loss = 1.1786e-03, PNorm = 90.4912, GNorm = 0.6479, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.443814
Epoch 75
Validation binary_cross_entropy = 0.447990
Epoch 76
Validation binary_cross_entropy = 0.464618
Epoch 77
Validation binary_cross_entropy = 0.484827
Epoch 78
Validation binary_cross_entropy = 0.486043
Epoch 79
Loss = 7.7473e-04, PNorm = 90.6437, GNorm = 0.5523, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.519574
Epoch 80
Validation binary_cross_entropy = 0.528850
Epoch 81
Validation binary_cross_entropy = 0.505424
Epoch 82
Validation binary_cross_entropy = 0.516289
Epoch 83
Validation binary_cross_entropy = 0.538924
Epoch 84
Loss = 4.5572e-04, PNorm = 90.7809, GNorm = 0.4448, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.578007
Epoch 85
Validation binary_cross_entropy = 0.606570
Epoch 86
Validation binary_cross_entropy = 0.592012
Epoch 87
Validation binary_cross_entropy = 0.583925
Epoch 88
Validation binary_cross_entropy = 0.581751
Epoch 89
Loss = 3.5738e-03, PNorm = 90.9246, GNorm = 2.3758, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.566586
Epoch 90
Validation binary_cross_entropy = 0.565385
Epoch 91
Validation binary_cross_entropy = 0.585183
Epoch 92
Validation binary_cross_entropy = 0.537406
Epoch 93
Validation binary_cross_entropy = 0.516145
Epoch 94
Loss = 6.2375e-04, PNorm = 91.0863, GNorm = 1.6607, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.550072
Epoch 95
Validation binary_cross_entropy = 0.561155
Epoch 96
Validation binary_cross_entropy = 0.619346
Epoch 97
Validation binary_cross_entropy = 0.650232
Epoch 98
Validation binary_cross_entropy = 0.673423
Epoch 99
Loss = 5.4545e-04, PNorm = 91.2595, GNorm = 0.8885, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.685644
Epoch 100
Validation binary_cross_entropy = 0.633494
Epoch 101
Validation binary_cross_entropy = 0.691513
Epoch 102
Validation binary_cross_entropy = 0.641267
Epoch 103
Validation binary_cross_entropy = 0.630371
Epoch 104
Loss = 2.5590e-04, PNorm = 91.3928, GNorm = 0.4309, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.692018
Epoch 105
Validation binary_cross_entropy = 0.716323
Epoch 106
Validation binary_cross_entropy = 0.644464
Epoch 107
Validation binary_cross_entropy = 0.622500
Epoch 108
Validation binary_cross_entropy = 0.621638
Epoch 109
Loss = 5.2984e-04, PNorm = 91.5095, GNorm = 1.4977, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.635684
Epoch 110
Validation binary_cross_entropy = 0.691536
Epoch 111
Validation binary_cross_entropy = 0.698249
Epoch 112
Validation binary_cross_entropy = 0.684186
Epoch 113
Validation binary_cross_entropy = 0.648790
Epoch 114
Loss = 1.6046e-04, PNorm = 91.6249, GNorm = 0.1489, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.607320
Epoch 115
Validation binary_cross_entropy = 0.603527
Epoch 116
Validation binary_cross_entropy = 0.613275
Epoch 117
Validation binary_cross_entropy = 0.613238
Epoch 118
Validation binary_cross_entropy = 0.646981
Epoch 119
Loss = 8.4238e-04, PNorm = 91.7153, GNorm = 1.6825, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.687580
Epoch 120
Validation binary_cross_entropy = 0.704185
Epoch 121
Validation binary_cross_entropy = 0.642147
Epoch 122
Validation binary_cross_entropy = 0.705724
Epoch 123
Validation binary_cross_entropy = 0.714571
Epoch 124
Loss = 7.2806e-04, PNorm = 91.8613, GNorm = 4.2929, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.647674
Epoch 125
Validation binary_cross_entropy = 0.604323
Epoch 126
Validation binary_cross_entropy = 0.578458
Epoch 127
Validation binary_cross_entropy = 0.549540
Epoch 128
Validation binary_cross_entropy = 0.511135
Epoch 129
Loss = 2.3029e-03, PNorm = 92.0954, GNorm = 2.0906, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.537644
Epoch 130
Validation binary_cross_entropy = 0.593348
Epoch 131
Validation binary_cross_entropy = 0.588117
Epoch 132
Validation binary_cross_entropy = 0.598202
Epoch 133
Validation binary_cross_entropy = 0.600317
Epoch 134
Loss = 3.8126e-04, PNorm = 92.3290, GNorm = 0.2216, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.593239
Epoch 135
Validation binary_cross_entropy = 0.637925
Epoch 136
Validation binary_cross_entropy = 0.625984
Epoch 137
Validation binary_cross_entropy = 0.590025
Epoch 138
Validation binary_cross_entropy = 0.572115
Epoch 139
Loss = 3.1594e-04, PNorm = 92.5580, GNorm = 1.0504, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.567129
Epoch 140
Validation binary_cross_entropy = 0.632510
Epoch 141
Validation binary_cross_entropy = 0.716250
Epoch 142
Validation binary_cross_entropy = 0.678619
Epoch 143
Validation binary_cross_entropy = 0.650516
Epoch 144
Loss = 9.8773e-04, PNorm = 92.7760, GNorm = 2.6414, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.606092
Epoch 145
Validation binary_cross_entropy = 0.608028
Epoch 146
Validation binary_cross_entropy = 0.620270
Epoch 147
Validation binary_cross_entropy = 0.638262
Epoch 148
Validation binary_cross_entropy = 0.647972
Epoch 149
Loss = 1.1680e-03, PNorm = 92.9544, GNorm = 2.1062, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.638059
Epoch 150
Validation binary_cross_entropy = 0.668786
Epoch 151
Validation binary_cross_entropy = 0.657510
Epoch 152
Validation binary_cross_entropy = 0.652201
Epoch 153
Validation binary_cross_entropy = 0.658323
Epoch 154
Loss = 6.1881e-04, PNorm = 93.0900, GNorm = 1.1076, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.668024
Epoch 155
Validation binary_cross_entropy = 0.675244
Epoch 156
Validation binary_cross_entropy = 0.703130
Epoch 157
Validation binary_cross_entropy = 0.720605
Epoch 158
Validation binary_cross_entropy = 0.675818
Epoch 159
Loss = 1.7026e-04, PNorm = 93.1683, GNorm = 0.1767, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.641025
Epoch 160
Validation binary_cross_entropy = 0.640253
Epoch 161
Validation binary_cross_entropy = 0.639319
Epoch 162
Validation binary_cross_entropy = 0.671491
Epoch 163
Validation binary_cross_entropy = 0.684477
Epoch 164
Loss = 3.0854e-04, PNorm = 93.2749, GNorm = 0.4148, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.700187
Epoch 165
Validation binary_cross_entropy = 0.732750
Epoch 166
Validation binary_cross_entropy = 0.709008
Epoch 167
Validation binary_cross_entropy = 0.699825
Epoch 168
Validation binary_cross_entropy = 0.704353
Epoch 169
Loss = 3.0371e-04, PNorm = 93.3740, GNorm = 0.4293, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.722122
Epoch 170
Validation binary_cross_entropy = 0.737552
Epoch 171
Validation binary_cross_entropy = 0.749189
Epoch 172
Validation binary_cross_entropy = 0.758705
Epoch 173
Validation binary_cross_entropy = 0.752023
Epoch 174
Loss = 1.4537e-04, PNorm = 93.4440, GNorm = 0.3347, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.748062
Epoch 175
Validation binary_cross_entropy = 0.750677
Epoch 176
Validation binary_cross_entropy = 0.761796
Epoch 177
Validation binary_cross_entropy = 0.781019
Epoch 178
Validation binary_cross_entropy = 0.794493
Epoch 179
Loss = 2.5419e-04, PNorm = 93.5025, GNorm = 1.3776, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.812118
Epoch 180
Validation binary_cross_entropy = 0.824223
Epoch 181
Validation binary_cross_entropy = 0.832418
Epoch 182
Validation binary_cross_entropy = 0.854799
Epoch 183
Validation binary_cross_entropy = 0.829879
Epoch 184
Loss = 3.3461e-04, PNorm = 93.5663, GNorm = 0.8149, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.786004
Epoch 185
Validation binary_cross_entropy = 0.761004
Epoch 186
Validation binary_cross_entropy = 0.766461
Epoch 187
Validation binary_cross_entropy = 0.766359
Epoch 188
Validation binary_cross_entropy = 0.776498
Epoch 189
Loss = 1.4740e-04, PNorm = 93.6533, GNorm = 0.4137, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.791018
Epoch 190
Validation binary_cross_entropy = 0.796301
Epoch 191
Validation binary_cross_entropy = 0.810033
Epoch 192
Validation binary_cross_entropy = 0.837547
Epoch 193
Validation binary_cross_entropy = 0.871518
Epoch 194
Loss = 3.6876e-04, PNorm = 93.7275, GNorm = 1.4315, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.879832
Epoch 195
Validation binary_cross_entropy = 0.855976
Epoch 196
Validation binary_cross_entropy = 0.833889
Epoch 197
Validation binary_cross_entropy = 0.819192
Epoch 198
Validation binary_cross_entropy = 0.811363
Epoch 199
Loss = 1.3987e-05, PNorm = 93.8039, GNorm = 0.0065, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.807811
Epoch 200
Validation binary_cross_entropy = 0.803777
Epoch 201
Validation binary_cross_entropy = 0.804203
Epoch 202
Validation binary_cross_entropy = 0.808198
Epoch 203
Validation binary_cross_entropy = 0.814062
Epoch 204
Loss = 8.3065e-05, PNorm = 93.8517, GNorm = 0.3784, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.800227
Epoch 205
Validation binary_cross_entropy = 0.793490
Epoch 206
Validation binary_cross_entropy = 0.798914
Epoch 207
Validation binary_cross_entropy = 0.807355
Epoch 208
Validation binary_cross_entropy = 0.819840
Epoch 209
Loss = 1.4815e-04, PNorm = 93.9070, GNorm = 1.2173, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.821039
Epoch 210
Validation binary_cross_entropy = 0.822286
Epoch 211
Validation binary_cross_entropy = 0.825741
Epoch 212
Validation binary_cross_entropy = 0.828554
Epoch 213
Validation binary_cross_entropy = 0.829225
Epoch 214
Loss = 1.5737e-05, PNorm = 93.9447, GNorm = 0.0346, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.834177
Epoch 215
Validation binary_cross_entropy = 0.839814
Epoch 216
Validation binary_cross_entropy = 0.847324
Epoch 217
Validation binary_cross_entropy = 0.866911
Epoch 218
Validation binary_cross_entropy = 0.887325
Epoch 219
Loss = 1.8698e-05, PNorm = 93.9818, GNorm = 0.0287, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.904434
Epoch 220
Validation binary_cross_entropy = 0.908399
Epoch 221
Validation binary_cross_entropy = 0.911090
Epoch 222
Validation binary_cross_entropy = 0.908224
Epoch 223
Validation binary_cross_entropy = 0.896739
Epoch 224
Loss = 4.4590e-04, PNorm = 94.0137, GNorm = 2.6380, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.950817
Epoch 225
Validation binary_cross_entropy = 1.057750
Epoch 226
Validation binary_cross_entropy = 0.938058
Epoch 227
Validation binary_cross_entropy = 0.836636
Epoch 228
Validation binary_cross_entropy = 0.805221
Epoch 229
Loss = 7.8457e-05, PNorm = 94.0890, GNorm = 0.3695, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.784157
Epoch 230
Validation binary_cross_entropy = 0.756419
Epoch 231
Validation binary_cross_entropy = 0.766365
Epoch 232
Validation binary_cross_entropy = 0.800620
Epoch 233
Validation binary_cross_entropy = 0.839300
Epoch 234
Loss = 1.3790e-04, PNorm = 94.2161, GNorm = 0.3202, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.861748
Epoch 235
Validation binary_cross_entropy = 0.829483
Epoch 236
Validation binary_cross_entropy = 0.822348
Epoch 237
Validation binary_cross_entropy = 0.810922
Epoch 238
Validation binary_cross_entropy = 0.830042
Epoch 239
Loss = 9.6339e-04, PNorm = 94.3312, GNorm = 2.0841, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.855367
Epoch 240
Validation binary_cross_entropy = 0.854784
Epoch 241
Validation binary_cross_entropy = 0.781292
Epoch 242
Validation binary_cross_entropy = 0.748713
Epoch 243
Validation binary_cross_entropy = 0.771194
Epoch 244
Loss = 1.6678e-04, PNorm = 94.4532, GNorm = 1.0248, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.803727
Epoch 245
Validation binary_cross_entropy = 0.819728
Epoch 246
Validation binary_cross_entropy = 0.843967
Epoch 247
Validation binary_cross_entropy = 0.850189
Epoch 248
Validation binary_cross_entropy = 0.843440
Epoch 249
Loss = 9.0480e-05, PNorm = 94.5960, GNorm = 0.1893, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.834671
Epoch 250
Validation binary_cross_entropy = 0.838724
Epoch 251
Validation binary_cross_entropy = 0.858331
Epoch 252
Validation binary_cross_entropy = 0.868641
Epoch 253
Validation binary_cross_entropy = 0.858591
Epoch 254
Loss = 6.6687e-04, PNorm = 94.6926, GNorm = 1.2565, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.844837
Epoch 255
Validation binary_cross_entropy = 0.838081
Epoch 256
Validation binary_cross_entropy = 0.834804
Epoch 257
Validation binary_cross_entropy = 0.836097
Epoch 258
Validation binary_cross_entropy = 0.843506
Epoch 259
Loss = 2.6029e-04, PNorm = 94.7623, GNorm = 0.0491, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.881919
Epoch 260
Validation binary_cross_entropy = 0.916316
Epoch 261
Validation binary_cross_entropy = 0.939974
Epoch 262
Validation binary_cross_entropy = 0.945238
Epoch 263
Validation binary_cross_entropy = 0.941472
Epoch 264
Loss = 2.3020e-04, PNorm = 94.8268, GNorm = 0.1892, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.888051
Epoch 265
Validation binary_cross_entropy = 0.849472
Epoch 266
Validation binary_cross_entropy = 0.873039
Epoch 267
Validation binary_cross_entropy = 0.967205
Epoch 268
Validation binary_cross_entropy = 0.911632
Epoch 269
Loss = 8.0521e-05, PNorm = 94.9134, GNorm = 0.2320, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.776335
Epoch 270
Validation binary_cross_entropy = 0.718891
Epoch 271
Validation binary_cross_entropy = 0.691091
Epoch 272
Validation binary_cross_entropy = 0.673030
Epoch 273
Validation binary_cross_entropy = 0.702329
Epoch 274
Loss = 2.4436e-05, PNorm = 94.9736, GNorm = 0.1107, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.736470
Epoch 275
Validation binary_cross_entropy = 0.780050
Epoch 276
Validation binary_cross_entropy = 0.812332
Epoch 277
Validation binary_cross_entropy = 0.791375
Epoch 278
Validation binary_cross_entropy = 0.725139
Epoch 279
Loss = 6.1905e-04, PNorm = 95.0464, GNorm = 0.7627, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.731615
Epoch 280
Validation binary_cross_entropy = 0.744465
Epoch 281
Validation binary_cross_entropy = 0.746221
Epoch 282
Validation binary_cross_entropy = 0.752162
Epoch 283
Validation binary_cross_entropy = 0.760111
Epoch 284
Loss = 5.5985e-04, PNorm = 95.1443, GNorm = 1.3081, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.811415
Epoch 285
Validation binary_cross_entropy = 0.886964
Epoch 286
Validation binary_cross_entropy = 0.936297
Epoch 287
Validation binary_cross_entropy = 0.869791
Epoch 288
Validation binary_cross_entropy = 0.823656
Epoch 289
Loss = 2.5342e-04, PNorm = 95.2322, GNorm = 0.8105, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.794880
Epoch 290
Validation binary_cross_entropy = 0.777026
Epoch 291
Validation binary_cross_entropy = 0.765416
Epoch 292
Validation binary_cross_entropy = 0.759409
Epoch 293
Validation binary_cross_entropy = 0.766134
Epoch 294
Loss = 1.3423e-04, PNorm = 95.3304, GNorm = 0.8326, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.778995
Epoch 295
Validation binary_cross_entropy = 0.750629
Epoch 296
Validation binary_cross_entropy = 0.731908
Epoch 297
Validation binary_cross_entropy = 0.721274
Epoch 298
Validation binary_cross_entropy = 0.715700
Epoch 299
Loss = 3.6394e-05, PNorm = 95.4114, GNorm = 0.1601, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.715718
Model 0 best validation binary_cross_entropy = 0.277389 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.240793
Ensemble test binary_cross_entropy = 0.240793
Fold 2
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_2',
 'save_smiles_splits': False,
 'seed': 2,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.281793
Best score so far is 0.281793 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.403612
Epoch 2
Validation binary_cross_entropy = 0.493063
Epoch 3
Validation binary_cross_entropy = 0.301732
Epoch 4
Loss = 1.0647e-02, PNorm = 87.7246, GNorm = 3.3057, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.463091
Epoch 5
Validation binary_cross_entropy = 0.418105
Epoch 6
Validation binary_cross_entropy = 0.276921
Best score so far is 0.276921 at epoch 6 
Epoch 7
Validation binary_cross_entropy = 0.693777
Epoch 8
Validation binary_cross_entropy = 0.315226
Epoch 9
Loss = 1.0306e-02, PNorm = 88.0306, GNorm = 4.5072, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.305550
Epoch 10
Validation binary_cross_entropy = 0.498948
Epoch 11
Validation binary_cross_entropy = 0.340429
Epoch 12
Validation binary_cross_entropy = 0.338112
Epoch 13
Validation binary_cross_entropy = 0.468103
Epoch 14
Loss = 4.3512e-03, PNorm = 88.3617, GNorm = 1.0154, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.447689
Epoch 15
Validation binary_cross_entropy = 0.411397
Epoch 16
Validation binary_cross_entropy = 0.436997
Epoch 17
Validation binary_cross_entropy = 0.423390
Epoch 18
Validation binary_cross_entropy = 0.410353
Epoch 19
Loss = 2.7522e-03, PNorm = 88.6134, GNorm = 1.1789, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.677484
Epoch 20
Validation binary_cross_entropy = 0.414919
Epoch 21
Validation binary_cross_entropy = 0.439105
Epoch 22
Validation binary_cross_entropy = 0.489090
Epoch 23
Validation binary_cross_entropy = 0.665297
Epoch 24
Loss = 4.9706e-03, PNorm = 88.8156, GNorm = 1.4308, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.550978
Epoch 25
Validation binary_cross_entropy = 0.482781
Epoch 26
Validation binary_cross_entropy = 0.412870
Epoch 27
Validation binary_cross_entropy = 0.419883
Epoch 28
Validation binary_cross_entropy = 0.391630
Epoch 29
Loss = 5.3577e-03, PNorm = 89.0032, GNorm = 2.0975, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.357337
Epoch 30
Validation binary_cross_entropy = 0.349810
Epoch 31
Validation binary_cross_entropy = 0.379013
Epoch 32
Validation binary_cross_entropy = 0.373011
Epoch 33
Validation binary_cross_entropy = 0.387852
Epoch 34
Loss = 4.3269e-03, PNorm = 89.1692, GNorm = 1.8271, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.396114
Epoch 35
Validation binary_cross_entropy = 0.367048
Epoch 36
Validation binary_cross_entropy = 0.364145
Epoch 37
Validation binary_cross_entropy = 0.355339
Epoch 38
Validation binary_cross_entropy = 0.450232
Epoch 39
Loss = 3.3741e-03, PNorm = 89.3391, GNorm = 2.4851, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.420518
Epoch 40
Validation binary_cross_entropy = 0.379203
Epoch 41
Validation binary_cross_entropy = 0.382411
Epoch 42
Validation binary_cross_entropy = 0.379718
Epoch 43
Validation binary_cross_entropy = 0.396908
Epoch 44
Loss = 3.0060e-03, PNorm = 89.5087, GNorm = 2.5295, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.387709
Epoch 45
Validation binary_cross_entropy = 0.403969
Epoch 46
Validation binary_cross_entropy = 0.425788
Epoch 47
Validation binary_cross_entropy = 0.452996
Epoch 48
Validation binary_cross_entropy = 0.464159
Epoch 49
Loss = 2.0091e-03, PNorm = 89.6565, GNorm = 1.2948, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.432618
Epoch 50
Validation binary_cross_entropy = 0.413613
Epoch 51
Validation binary_cross_entropy = 0.416374
Epoch 52
Validation binary_cross_entropy = 0.458841
Epoch 53
Validation binary_cross_entropy = 0.443452
Epoch 54
Loss = 1.0260e-03, PNorm = 89.8151, GNorm = 0.3136, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.487067
Epoch 55
Validation binary_cross_entropy = 0.496434
Epoch 56
Validation binary_cross_entropy = 0.517439
Epoch 57
Validation binary_cross_entropy = 0.469414
Epoch 58
Validation binary_cross_entropy = 0.489428
Epoch 59
Loss = 2.6086e-03, PNorm = 90.0190, GNorm = 2.8014, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.460098
Epoch 60
Validation binary_cross_entropy = 0.449449
Epoch 61
Validation binary_cross_entropy = 0.491378
Epoch 62
Validation binary_cross_entropy = 0.502670
Epoch 63
Validation binary_cross_entropy = 0.484570
Epoch 64
Loss = 6.7833e-04, PNorm = 90.2170, GNorm = 0.5358, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.503020
Epoch 65
Validation binary_cross_entropy = 0.657557
Epoch 66
Validation binary_cross_entropy = 0.547403
Epoch 67
Validation binary_cross_entropy = 0.477646
Epoch 68
Validation binary_cross_entropy = 0.536301
Epoch 69
Loss = 1.5171e-03, PNorm = 90.4064, GNorm = 0.8607, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.522542
Epoch 70
Validation binary_cross_entropy = 0.553201
Epoch 71
Validation binary_cross_entropy = 0.581726
Epoch 72
Validation binary_cross_entropy = 0.595669
Epoch 73
Validation binary_cross_entropy = 0.591656
Epoch 74
Loss = 5.5683e-04, PNorm = 90.5708, GNorm = 0.8147, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.575253
Epoch 75
Validation binary_cross_entropy = 0.559713
Epoch 76
Validation binary_cross_entropy = 0.551024
Epoch 77
Validation binary_cross_entropy = 0.554878
Epoch 78
Validation binary_cross_entropy = 0.578028
Epoch 79
Loss = 2.0139e-03, PNorm = 90.7123, GNorm = 1.8283, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.621783
Epoch 80
Validation binary_cross_entropy = 0.617234
Epoch 81
Validation binary_cross_entropy = 0.617752
Epoch 82
Validation binary_cross_entropy = 0.615274
Epoch 83
Validation binary_cross_entropy = 0.612569
Epoch 84
Loss = 1.5533e-03, PNorm = 90.8655, GNorm = 2.2320, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.620530
Epoch 85
Validation binary_cross_entropy = 0.639045
Epoch 86
Validation binary_cross_entropy = 0.655260
Epoch 87
Validation binary_cross_entropy = 0.670421
Epoch 88
Validation binary_cross_entropy = 0.688591
Epoch 89
Loss = 2.2460e-03, PNorm = 91.0197, GNorm = 1.2290, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.706596
Epoch 90
Validation binary_cross_entropy = 0.679071
Epoch 91
Validation binary_cross_entropy = 0.626882
Epoch 92
Validation binary_cross_entropy = 0.635484
Epoch 93
Validation binary_cross_entropy = 0.602494
Epoch 94
Loss = 1.9041e-03, PNorm = 91.2625, GNorm = 1.5480, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.600340
Epoch 95
Validation binary_cross_entropy = 0.574530
Epoch 96
Validation binary_cross_entropy = 0.519637
Epoch 97
Validation binary_cross_entropy = 0.544383
Epoch 98
Validation binary_cross_entropy = 0.529881
Epoch 99
Loss = 1.6924e-03, PNorm = 91.5356, GNorm = 1.9259, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.527118
Epoch 100
Validation binary_cross_entropy = 0.548038
Epoch 101
Validation binary_cross_entropy = 0.566955
Epoch 102
Validation binary_cross_entropy = 0.560993
Epoch 103
Validation binary_cross_entropy = 0.561913
Epoch 104
Loss = 2.9224e-03, PNorm = 91.7475, GNorm = 1.6957, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.592071
Epoch 105
Validation binary_cross_entropy = 0.613422
Epoch 106
Validation binary_cross_entropy = 0.588223
Epoch 107
Validation binary_cross_entropy = 0.573080
Epoch 108
Validation binary_cross_entropy = 0.587968
Epoch 109
Loss = 9.5960e-04, PNorm = 91.9130, GNorm = 1.3484, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.617565
Epoch 110
Validation binary_cross_entropy = 0.645427
Epoch 111
Validation binary_cross_entropy = 0.643113
Epoch 112
Validation binary_cross_entropy = 0.600266
Epoch 113
Validation binary_cross_entropy = 0.588006
Epoch 114
Loss = 5.7289e-04, PNorm = 92.0562, GNorm = 1.0585, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.611965
Epoch 115
Validation binary_cross_entropy = 0.650817
Epoch 116
Validation binary_cross_entropy = 0.791527
Epoch 117
Validation binary_cross_entropy = 0.770103
Epoch 118
Validation binary_cross_entropy = 0.710623
Epoch 119
Loss = 5.6271e-04, PNorm = 92.1909, GNorm = 0.8564, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.696701
Epoch 120
Validation binary_cross_entropy = 0.669312
Epoch 121
Validation binary_cross_entropy = 0.724794
Epoch 122
Validation binary_cross_entropy = 0.674915
Epoch 123
Validation binary_cross_entropy = 0.609385
Epoch 124
Loss = 4.9599e-04, PNorm = 92.3418, GNorm = 1.2509, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.607303
Epoch 125
Validation binary_cross_entropy = 0.608151
Epoch 126
Validation binary_cross_entropy = 0.645622
Epoch 127
Validation binary_cross_entropy = 0.680829
Epoch 128
Validation binary_cross_entropy = 0.667306
Epoch 129
Loss = 1.2185e-03, PNorm = 92.4772, GNorm = 0.1631, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.672567
Epoch 130
Validation binary_cross_entropy = 0.686632
Epoch 131
Validation binary_cross_entropy = 0.701610
Epoch 132
Validation binary_cross_entropy = 0.676216
Epoch 133
Validation binary_cross_entropy = 0.656348
Epoch 134
Loss = 4.0110e-04, PNorm = 92.6046, GNorm = 0.9863, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.676350
Epoch 135
Validation binary_cross_entropy = 0.677745
Epoch 136
Validation binary_cross_entropy = 0.713181
Epoch 137
Validation binary_cross_entropy = 0.737014
Epoch 138
Validation binary_cross_entropy = 0.752006
Epoch 139
Loss = 6.7002e-04, PNorm = 92.7182, GNorm = 1.4494, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.716833
Epoch 140
Validation binary_cross_entropy = 0.682614
Epoch 141
Validation binary_cross_entropy = 0.664280
Epoch 142
Validation binary_cross_entropy = 0.638313
Epoch 143
Validation binary_cross_entropy = 0.626795
Epoch 144
Loss = 1.6531e-03, PNorm = 92.8440, GNorm = 0.5146, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.628556
Epoch 145
Validation binary_cross_entropy = 0.656376
Epoch 146
Validation binary_cross_entropy = 0.665288
Epoch 147
Validation binary_cross_entropy = 0.657287
Epoch 148
Validation binary_cross_entropy = 0.694612
Epoch 149
Loss = 1.6525e-04, PNorm = 92.9671, GNorm = 0.1287, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.774226
Epoch 150
Validation binary_cross_entropy = 0.819626
Epoch 151
Validation binary_cross_entropy = 0.801866
Epoch 152
Validation binary_cross_entropy = 0.764544
Epoch 153
Validation binary_cross_entropy = 0.755912
Epoch 154
Loss = 1.6665e-04, PNorm = 93.0927, GNorm = 0.3104, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.757529
Epoch 155
Validation binary_cross_entropy = 0.766080
Epoch 156
Validation binary_cross_entropy = 0.745633
Epoch 157
Validation binary_cross_entropy = 0.722702
Epoch 158
Validation binary_cross_entropy = 0.717459
Epoch 159
Loss = 7.8889e-04, PNorm = 93.2170, GNorm = 0.5346, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.751515
Epoch 160
Validation binary_cross_entropy = 0.793829
Epoch 161
Validation binary_cross_entropy = 0.832789
Epoch 162
Validation binary_cross_entropy = 0.839533
Epoch 163
Validation binary_cross_entropy = 0.796183
Epoch 164
Loss = 2.7638e-04, PNorm = 93.3368, GNorm = 0.1821, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.754545
Epoch 165
Validation binary_cross_entropy = 0.755061
Epoch 166
Validation binary_cross_entropy = 0.759313
Epoch 167
Validation binary_cross_entropy = 0.780294
Epoch 168
Validation binary_cross_entropy = 0.809208
Epoch 169
Loss = 5.8454e-04, PNorm = 93.4852, GNorm = 0.4740, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.841392
Epoch 170
Validation binary_cross_entropy = 0.859451
Epoch 171
Validation binary_cross_entropy = 0.850403
Epoch 172
Validation binary_cross_entropy = 0.841811
Epoch 173
Validation binary_cross_entropy = 0.837509
Epoch 174
Loss = 6.0607e-04, PNorm = 93.5828, GNorm = 0.0299, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.827916
Epoch 175
Validation binary_cross_entropy = 0.828079
Epoch 176
Validation binary_cross_entropy = 0.844606
Epoch 177
Validation binary_cross_entropy = 0.869603
Epoch 178
Validation binary_cross_entropy = 0.885765
Epoch 179
Loss = 3.0096e-04, PNorm = 93.6704, GNorm = 0.3688, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.821553
Epoch 180
Validation binary_cross_entropy = 0.793204
Epoch 181
Validation binary_cross_entropy = 0.770820
Epoch 182
Validation binary_cross_entropy = 0.764951
Epoch 183
Validation binary_cross_entropy = 0.772289
Epoch 184
Loss = 6.8555e-04, PNorm = 93.7353, GNorm = 0.0439, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.797664
Epoch 185
Validation binary_cross_entropy = 0.852802
Epoch 186
Validation binary_cross_entropy = 0.860651
Epoch 187
Validation binary_cross_entropy = 0.839597
Epoch 188
Validation binary_cross_entropy = 0.841985
Epoch 189
Loss = 5.6058e-04, PNorm = 93.8043, GNorm = 1.3159, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.844831
Epoch 190
Validation binary_cross_entropy = 0.824321
Epoch 191
Validation binary_cross_entropy = 0.787432
Epoch 192
Validation binary_cross_entropy = 0.785156
Epoch 193
Validation binary_cross_entropy = 0.780124
Epoch 194
Loss = 1.0483e-04, PNorm = 93.8656, GNorm = 0.2976, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.763229
Epoch 195
Validation binary_cross_entropy = 0.754327
Epoch 196
Validation binary_cross_entropy = 0.789345
Epoch 197
Validation binary_cross_entropy = 0.831545
Epoch 198
Validation binary_cross_entropy = 0.868928
Epoch 199
Loss = 6.5784e-05, PNorm = 93.9613, GNorm = 0.1410, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.886991
Epoch 200
Validation binary_cross_entropy = 0.835568
Epoch 201
Validation binary_cross_entropy = 0.817863
Epoch 202
Validation binary_cross_entropy = 0.812827
Epoch 203
Validation binary_cross_entropy = 0.816147
Epoch 204
Loss = 5.8089e-04, PNorm = 94.0451, GNorm = 3.4016, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.858082
Epoch 205
Validation binary_cross_entropy = 0.937333
Epoch 206
Validation binary_cross_entropy = 0.991610
Epoch 207
Validation binary_cross_entropy = 0.966539
Epoch 208
Validation binary_cross_entropy = 0.856771
Epoch 209
Loss = 6.7081e-05, PNorm = 94.1957, GNorm = 0.0653, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.782030
Epoch 210
Validation binary_cross_entropy = 0.754399
Epoch 211
Validation binary_cross_entropy = 0.767448
Epoch 212
Validation binary_cross_entropy = 0.761160
Epoch 213
Validation binary_cross_entropy = 0.751604
Epoch 214
Loss = 1.2873e-04, PNorm = 94.3171, GNorm = 0.3544, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.768682
Epoch 215
Validation binary_cross_entropy = 0.798674
Epoch 216
Validation binary_cross_entropy = 0.828019
Epoch 217
Validation binary_cross_entropy = 0.836098
Epoch 218
Validation binary_cross_entropy = 0.823188
Epoch 219
Loss = 7.9369e-05, PNorm = 94.4101, GNorm = 0.1800, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.819418
Epoch 220
Validation binary_cross_entropy = 0.833099
Epoch 221
Validation binary_cross_entropy = 0.847887
Epoch 222
Validation binary_cross_entropy = 0.882349
Epoch 223
Validation binary_cross_entropy = 0.938232
Epoch 224
Loss = 4.3460e-04, PNorm = 94.4874, GNorm = 1.9904, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.933221
Epoch 225
Validation binary_cross_entropy = 0.908532
Epoch 226
Validation binary_cross_entropy = 0.889363
Epoch 227
Validation binary_cross_entropy = 0.862105
Epoch 228
Validation binary_cross_entropy = 0.839380
Epoch 229
Loss = 7.3490e-04, PNorm = 94.5532, GNorm = 1.9900, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.810370
Epoch 230
Validation binary_cross_entropy = 0.770905
Epoch 231
Validation binary_cross_entropy = 0.755648
Epoch 232
Validation binary_cross_entropy = 0.785386
Epoch 233
Validation binary_cross_entropy = 0.787026
Epoch 234
Loss = 1.4306e-04, PNorm = 94.7041, GNorm = 0.4142, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.770870
Epoch 235
Validation binary_cross_entropy = 0.801679
Epoch 236
Validation binary_cross_entropy = 0.849627
Epoch 237
Validation binary_cross_entropy = 0.883979
Epoch 238
Validation binary_cross_entropy = 0.907172
Epoch 239
Loss = 8.2991e-04, PNorm = 94.8385, GNorm = 1.5118, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.796667
Epoch 240
Validation binary_cross_entropy = 0.720532
Epoch 241
Validation binary_cross_entropy = 0.722268
Epoch 242
Validation binary_cross_entropy = 0.750758
Epoch 243
Validation binary_cross_entropy = 0.719899
Epoch 244
Loss = 2.4534e-04, PNorm = 94.9518, GNorm = 0.0876, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.799447
Epoch 245
Validation binary_cross_entropy = 0.925585
Epoch 246
Validation binary_cross_entropy = 0.892472
Epoch 247
Validation binary_cross_entropy = 0.821283
Epoch 248
Validation binary_cross_entropy = 0.760831
Epoch 249
Loss = 2.9439e-04, PNorm = 95.0837, GNorm = 0.7985, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.738784
Epoch 250
Validation binary_cross_entropy = 0.730970
Epoch 251
Validation binary_cross_entropy = 0.733148
Epoch 252
Validation binary_cross_entropy = 0.750101
Epoch 253
Validation binary_cross_entropy = 0.773176
Epoch 254
Loss = 7.7423e-05, PNorm = 95.1852, GNorm = 0.1404, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.787851
Epoch 255
Validation binary_cross_entropy = 0.782473
Epoch 256
Validation binary_cross_entropy = 0.831131
Epoch 257
Validation binary_cross_entropy = 0.845247
Epoch 258
Validation binary_cross_entropy = 0.839567
Epoch 259
Loss = 7.0967e-05, PNorm = 95.2589, GNorm = 0.0564, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.845608
Epoch 260
Validation binary_cross_entropy = 0.814529
Epoch 261
Validation binary_cross_entropy = 0.751745
Epoch 262
Validation binary_cross_entropy = 0.706294
Epoch 263
Validation binary_cross_entropy = 0.698135
Epoch 264
Loss = 6.7265e-05, PNorm = 95.3568, GNorm = 0.0706, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.724037
Epoch 265
Validation binary_cross_entropy = 0.739596
Epoch 266
Validation binary_cross_entropy = 0.717336
Epoch 267
Validation binary_cross_entropy = 0.710815
Epoch 268
Validation binary_cross_entropy = 0.718022
Epoch 269
Loss = 1.5120e-03, PNorm = 95.4833, GNorm = 1.5053, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.749202
Epoch 270
Validation binary_cross_entropy = 0.814163
Epoch 271
Validation binary_cross_entropy = 0.857854
Epoch 272
Validation binary_cross_entropy = 0.847834
Epoch 273
Validation binary_cross_entropy = 0.844886
Epoch 274
Loss = 9.6604e-05, PNorm = 95.5675, GNorm = 0.0707, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.821311
Epoch 275
Validation binary_cross_entropy = 0.809955
Epoch 276
Validation binary_cross_entropy = 0.814138
Epoch 277
Validation binary_cross_entropy = 0.824511
Epoch 278
Validation binary_cross_entropy = 0.845101
Epoch 279
Loss = 9.3251e-05, PNorm = 95.6240, GNorm = 0.4602, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.868115
Epoch 280
Validation binary_cross_entropy = 0.891105
Epoch 281
Validation binary_cross_entropy = 0.936068
Epoch 282
Validation binary_cross_entropy = 0.982905
Epoch 283
Validation binary_cross_entropy = 1.009127
Epoch 284
Loss = 2.1242e-04, PNorm = 95.6627, GNorm = 0.8162, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.022828
Epoch 285
Validation binary_cross_entropy = 1.017579
Epoch 286
Validation binary_cross_entropy = 0.994268
Epoch 287
Validation binary_cross_entropy = 0.957035
Epoch 288
Validation binary_cross_entropy = 0.929258
Epoch 289
Loss = 1.9829e-05, PNorm = 95.7070, GNorm = 0.0336, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.921032
Epoch 290
Validation binary_cross_entropy = 0.919797
Epoch 291
Validation binary_cross_entropy = 0.928856
Epoch 292
Validation binary_cross_entropy = 0.939783
Epoch 293
Validation binary_cross_entropy = 0.957925
Epoch 294
Loss = 3.1734e-05, PNorm = 95.7409, GNorm = 0.0152, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.983567
Epoch 295
Validation binary_cross_entropy = 0.988036
Epoch 296
Validation binary_cross_entropy = 0.976635
Epoch 297
Validation binary_cross_entropy = 0.967475
Epoch 298
Validation binary_cross_entropy = 0.959058
Epoch 299
Loss = 6.0371e-05, PNorm = 95.7580, GNorm = 0.3356, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.953909
Model 0 best validation binary_cross_entropy = 0.276921 on epoch 6
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.149826
Ensemble test binary_cross_entropy = 0.149826
Fold 3
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_3',
 'save_smiles_splits': False,
 'seed': 3,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.276116
Best score so far is 0.276116 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.345855
Epoch 2
Validation binary_cross_entropy = 0.334271
Epoch 3
Validation binary_cross_entropy = 0.519500
Epoch 4
Loss = 9.7888e-03, PNorm = 87.7218, GNorm = 2.4036, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.286657
Epoch 5
Validation binary_cross_entropy = 0.648542
Epoch 6
Validation binary_cross_entropy = 0.388202
Epoch 7
Validation binary_cross_entropy = 0.408460
Epoch 8
Validation binary_cross_entropy = 0.488442
Epoch 9
Loss = 7.8056e-03, PNorm = 88.0213, GNorm = 2.0356, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.340601
Epoch 10
Validation binary_cross_entropy = 0.445269
Epoch 11
Validation binary_cross_entropy = 0.341488
Epoch 12
Validation binary_cross_entropy = 0.424847
Epoch 13
Validation binary_cross_entropy = 0.355913
Epoch 14
Loss = 4.8671e-03, PNorm = 88.3381, GNorm = 1.9383, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.315930
Epoch 15
Validation binary_cross_entropy = 0.370351
Epoch 16
Validation binary_cross_entropy = 0.310198
Epoch 17
Validation binary_cross_entropy = 0.334027
Epoch 18
Validation binary_cross_entropy = 0.522317
Epoch 19
Loss = 3.6198e-03, PNorm = 88.5733, GNorm = 1.8506, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.395962
Epoch 20
Validation binary_cross_entropy = 0.365974
Epoch 21
Validation binary_cross_entropy = 0.366779
Epoch 22
Validation binary_cross_entropy = 0.450307
Epoch 23
Validation binary_cross_entropy = 0.351321
Epoch 24
Loss = 5.1227e-03, PNorm = 88.7694, GNorm = 4.3077, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.360099
Epoch 25
Validation binary_cross_entropy = 0.533449
Epoch 26
Validation binary_cross_entropy = 0.377003
Epoch 27
Validation binary_cross_entropy = 0.403463
Epoch 28
Validation binary_cross_entropy = 0.436025
Epoch 29
Loss = 3.9511e-03, PNorm = 88.9521, GNorm = 3.1628, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.415937
Epoch 30
Validation binary_cross_entropy = 0.360054
Epoch 31
Validation binary_cross_entropy = 0.369239
Epoch 32
Validation binary_cross_entropy = 0.380902
Epoch 33
Validation binary_cross_entropy = 0.399224
Epoch 34
Loss = 1.3081e-03, PNorm = 89.1281, GNorm = 0.3474, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.389463
Epoch 35
Validation binary_cross_entropy = 0.384002
Epoch 36
Validation binary_cross_entropy = 0.382225
Epoch 37
Validation binary_cross_entropy = 0.439550
Epoch 38
Validation binary_cross_entropy = 0.483401
Epoch 39
Loss = 9.7609e-04, PNorm = 89.2830, GNorm = 0.6720, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.470909
Epoch 40
Validation binary_cross_entropy = 0.463919
Epoch 41
Validation binary_cross_entropy = 0.514017
Epoch 42
Validation binary_cross_entropy = 0.464611
Epoch 43
Validation binary_cross_entropy = 0.411930
Epoch 44
Loss = 2.2342e-03, PNorm = 89.4405, GNorm = 1.2492, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.417627
Epoch 45
Validation binary_cross_entropy = 0.438532
Epoch 46
Validation binary_cross_entropy = 0.422220
Epoch 47
Validation binary_cross_entropy = 0.403674
Epoch 48
Validation binary_cross_entropy = 0.384471
Epoch 49
Loss = 2.4172e-03, PNorm = 89.5925, GNorm = 1.2648, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.382773
Epoch 50
Validation binary_cross_entropy = 0.392569
Epoch 51
Validation binary_cross_entropy = 0.432462
Epoch 52
Validation binary_cross_entropy = 0.448634
Epoch 53
Validation binary_cross_entropy = 0.495240
Epoch 54
Loss = 6.6586e-04, PNorm = 89.7295, GNorm = 1.0684, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.509299
Epoch 55
Validation binary_cross_entropy = 0.439138
Epoch 56
Validation binary_cross_entropy = 0.530179
Epoch 57
Validation binary_cross_entropy = 0.374767
Epoch 58
Validation binary_cross_entropy = 0.393163
Epoch 59
Loss = 2.0824e-03, PNorm = 89.8831, GNorm = 1.3268, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.380819
Epoch 60
Validation binary_cross_entropy = 0.366592
Epoch 61
Validation binary_cross_entropy = 0.383843
Epoch 62
Validation binary_cross_entropy = 0.393262
Epoch 63
Validation binary_cross_entropy = 0.382712
Epoch 64
Loss = 1.4556e-03, PNorm = 90.0691, GNorm = 1.3916, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.379858
Epoch 65
Validation binary_cross_entropy = 0.358544
Epoch 66
Validation binary_cross_entropy = 0.387464
Epoch 67
Validation binary_cross_entropy = 0.388437
Epoch 68
Validation binary_cross_entropy = 0.393756
Epoch 69
Loss = 1.4346e-03, PNorm = 90.2330, GNorm = 0.8128, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.405729
Epoch 70
Validation binary_cross_entropy = 0.421727
Epoch 71
Validation binary_cross_entropy = 0.441434
Epoch 72
Validation binary_cross_entropy = 0.459303
Epoch 73
Validation binary_cross_entropy = 0.502619
Epoch 74
Loss = 7.4855e-04, PNorm = 90.3955, GNorm = 1.1435, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.537897
Epoch 75
Validation binary_cross_entropy = 0.483256
Epoch 76
Validation binary_cross_entropy = 0.464449
Epoch 77
Validation binary_cross_entropy = 0.457349
Epoch 78
Validation binary_cross_entropy = 0.486122
Epoch 79
Loss = 1.2789e-03, PNorm = 90.5252, GNorm = 1.0075, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.481908
Epoch 80
Validation binary_cross_entropy = 0.512231
Epoch 81
Validation binary_cross_entropy = 0.522452
Epoch 82
Validation binary_cross_entropy = 0.527738
Epoch 83
Validation binary_cross_entropy = 0.508710
Epoch 84
Loss = 7.3212e-04, PNorm = 90.6620, GNorm = 1.0008, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.512349
Epoch 85
Validation binary_cross_entropy = 0.511626
Epoch 86
Validation binary_cross_entropy = 0.513821
Epoch 87
Validation binary_cross_entropy = 0.513427
Epoch 88
Validation binary_cross_entropy = 0.504166
Epoch 89
Loss = 1.2046e-03, PNorm = 90.7669, GNorm = 1.4295, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.496779
Epoch 90
Validation binary_cross_entropy = 0.497028
Epoch 91
Validation binary_cross_entropy = 0.499193
Epoch 92
Validation binary_cross_entropy = 0.505848
Epoch 93
Validation binary_cross_entropy = 0.511001
Epoch 94
Loss = 9.0898e-04, PNorm = 90.8755, GNorm = 0.5972, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.519173
Epoch 95
Validation binary_cross_entropy = 0.532197
Epoch 96
Validation binary_cross_entropy = 0.527183
Epoch 97
Validation binary_cross_entropy = 0.676141
Epoch 98
Validation binary_cross_entropy = 0.468636
Epoch 99
Loss = 4.6053e-04, PNorm = 90.9966, GNorm = 0.6791, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.467606
Epoch 100
Validation binary_cross_entropy = 0.471600
Epoch 101
Validation binary_cross_entropy = 0.440865
Epoch 102
Validation binary_cross_entropy = 0.485722
Epoch 103
Validation binary_cross_entropy = 0.539233
Epoch 104
Loss = 1.9294e-03, PNorm = 91.2057, GNorm = 1.3354, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.456667
Epoch 105
Validation binary_cross_entropy = 0.435086
Epoch 106
Validation binary_cross_entropy = 0.461277
Epoch 107
Validation binary_cross_entropy = 0.467417
Epoch 108
Validation binary_cross_entropy = 0.499514
Epoch 109
Loss = 4.3986e-04, PNorm = 91.3850, GNorm = 0.2684, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.529778
Epoch 110
Validation binary_cross_entropy = 0.526594
Epoch 111
Validation binary_cross_entropy = 0.519284
Epoch 112
Validation binary_cross_entropy = 0.528771
Epoch 113
Validation binary_cross_entropy = 0.520232
Epoch 114
Loss = 6.8165e-04, PNorm = 91.5220, GNorm = 0.1542, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.513576
Epoch 115
Validation binary_cross_entropy = 0.521066
Epoch 116
Validation binary_cross_entropy = 0.523654
Epoch 117
Validation binary_cross_entropy = 0.532288
Epoch 118
Validation binary_cross_entropy = 0.561191
Epoch 119
Loss = 6.1013e-04, PNorm = 91.6184, GNorm = 1.2142, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.570787
Epoch 120
Validation binary_cross_entropy = 0.617284
Epoch 121
Validation binary_cross_entropy = 0.664734
Epoch 122
Validation binary_cross_entropy = 0.690103
Epoch 123
Validation binary_cross_entropy = 0.674187
Epoch 124
Loss = 2.5948e-03, PNorm = 91.7285, GNorm = 1.1456, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.705306
Epoch 125
Validation binary_cross_entropy = 0.654546
Epoch 126
Validation binary_cross_entropy = 0.629905
Epoch 127
Validation binary_cross_entropy = 0.663790
Epoch 128
Validation binary_cross_entropy = 0.702064
Epoch 129
Loss = 3.2142e-03, PNorm = 91.8556, GNorm = 1.9678, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.669048
Epoch 130
Validation binary_cross_entropy = 0.650703
Epoch 131
Validation binary_cross_entropy = 0.682568
Epoch 132
Validation binary_cross_entropy = 0.692897
Epoch 133
Validation binary_cross_entropy = 0.635853
Epoch 134
Loss = 1.3632e-03, PNorm = 92.0152, GNorm = 1.4541, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.600179
Epoch 135
Validation binary_cross_entropy = 0.645058
Epoch 136
Validation binary_cross_entropy = 0.602274
Epoch 137
Validation binary_cross_entropy = 0.549447
Epoch 138
Validation binary_cross_entropy = 0.523573
Epoch 139
Loss = 2.0535e-04, PNorm = 92.2516, GNorm = 0.1826, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.521254
Epoch 140
Validation binary_cross_entropy = 0.527622
Epoch 141
Validation binary_cross_entropy = 0.553919
Epoch 142
Validation binary_cross_entropy = 0.607138
Epoch 143
Validation binary_cross_entropy = 0.537716
Epoch 144
Loss = 7.6851e-04, PNorm = 92.4060, GNorm = 1.1944, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.533110
Epoch 145
Validation binary_cross_entropy = 0.529798
Epoch 146
Validation binary_cross_entropy = 0.528234
Epoch 147
Validation binary_cross_entropy = 0.532857
Epoch 148
Validation binary_cross_entropy = 0.544920
Epoch 149
Loss = 6.2182e-04, PNorm = 92.5352, GNorm = 0.7990, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.554365
Epoch 150
Validation binary_cross_entropy = 0.557736
Epoch 151
Validation binary_cross_entropy = 0.572144
Epoch 152
Validation binary_cross_entropy = 0.594178
Epoch 153
Validation binary_cross_entropy = 0.603636
Epoch 154
Loss = 1.3434e-03, PNorm = 92.6322, GNorm = 1.2623, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.609015
Epoch 155
Validation binary_cross_entropy = 0.615088
Epoch 156
Validation binary_cross_entropy = 0.616924
Epoch 157
Validation binary_cross_entropy = 0.619099
Epoch 158
Validation binary_cross_entropy = 0.610851
Epoch 159
Loss = 9.7135e-04, PNorm = 92.7258, GNorm = 1.0072, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.599015
Epoch 160
Validation binary_cross_entropy = 0.583834
Epoch 161
Validation binary_cross_entropy = 0.580665
Epoch 162
Validation binary_cross_entropy = 0.585494
Epoch 163
Validation binary_cross_entropy = 0.617971
Epoch 164
Loss = 5.5664e-04, PNorm = 92.8121, GNorm = 0.2210, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.670562
Epoch 165
Validation binary_cross_entropy = 0.685171
Epoch 166
Validation binary_cross_entropy = 0.712307
Epoch 167
Validation binary_cross_entropy = 0.729465
Epoch 168
Validation binary_cross_entropy = 0.742227
Epoch 169
Loss = 1.4139e-03, PNorm = 92.8932, GNorm = 1.4585, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.736780
Epoch 170
Validation binary_cross_entropy = 0.737763
Epoch 171
Validation binary_cross_entropy = 0.714432
Epoch 172
Validation binary_cross_entropy = 0.689153
Epoch 173
Validation binary_cross_entropy = 0.712518
Epoch 174
Loss = 3.9844e-04, PNorm = 92.9963, GNorm = 0.6144, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.731851
Epoch 175
Validation binary_cross_entropy = 0.746972
Epoch 176
Validation binary_cross_entropy = 0.733954
Epoch 177
Validation binary_cross_entropy = 0.717403
Epoch 178
Validation binary_cross_entropy = 0.707083
Epoch 179
Loss = 3.2414e-04, PNorm = 93.1319, GNorm = 0.8124, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.685356
Epoch 180
Validation binary_cross_entropy = 0.663305
Epoch 181
Validation binary_cross_entropy = 0.650195
Epoch 182
Validation binary_cross_entropy = 0.648982
Epoch 183
Validation binary_cross_entropy = 0.647551
Epoch 184
Loss = 6.1955e-05, PNorm = 93.2541, GNorm = 0.2567, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.646030
Epoch 185
Validation binary_cross_entropy = 0.662235
Epoch 186
Validation binary_cross_entropy = 0.685846
Epoch 187
Validation binary_cross_entropy = 0.715581
Epoch 188
Validation binary_cross_entropy = 0.719420
Epoch 189
Loss = 1.6606e-04, PNorm = 93.3504, GNorm = 0.1276, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.691176
Epoch 190
Validation binary_cross_entropy = 0.662185
Epoch 191
Validation binary_cross_entropy = 0.635687
Epoch 192
Validation binary_cross_entropy = 0.632306
Epoch 193
Validation binary_cross_entropy = 0.644466
Epoch 194
Loss = 3.4513e-04, PNorm = 93.4296, GNorm = 0.0394, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.645326
Epoch 195
Validation binary_cross_entropy = 0.644030
Epoch 196
Validation binary_cross_entropy = 0.633982
Epoch 197
Validation binary_cross_entropy = 0.646191
Epoch 198
Validation binary_cross_entropy = 0.673145
Epoch 199
Loss = 6.8255e-04, PNorm = 93.5298, GNorm = 0.2925, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.737933
Epoch 200
Validation binary_cross_entropy = 0.813357
Epoch 201
Validation binary_cross_entropy = 0.807995
Epoch 202
Validation binary_cross_entropy = 0.781611
Epoch 203
Validation binary_cross_entropy = 0.726583
Epoch 204
Loss = 5.2609e-05, PNorm = 93.6343, GNorm = 0.1391, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.698679
Epoch 205
Validation binary_cross_entropy = 0.684902
Epoch 206
Validation binary_cross_entropy = 0.679962
Epoch 207
Validation binary_cross_entropy = 0.674899
Epoch 208
Validation binary_cross_entropy = 0.683952
Epoch 209
Loss = 1.7421e-04, PNorm = 93.7128, GNorm = 0.5714, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.694475
Epoch 210
Validation binary_cross_entropy = 0.689369
Epoch 211
Validation binary_cross_entropy = 0.695076
Epoch 212
Validation binary_cross_entropy = 0.707916
Epoch 213
Validation binary_cross_entropy = 0.718769
Epoch 214
Loss = 9.7030e-05, PNorm = 93.7629, GNorm = 0.5272, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.729837
Epoch 215
Validation binary_cross_entropy = 0.752251
Epoch 216
Validation binary_cross_entropy = 0.768662
Epoch 217
Validation binary_cross_entropy = 0.793636
Epoch 218
Validation binary_cross_entropy = 0.797648
Epoch 219
Loss = 2.2728e-04, PNorm = 93.8088, GNorm = 0.2429, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.734479
Epoch 220
Validation binary_cross_entropy = 0.687086
Epoch 221
Validation binary_cross_entropy = 0.663070
Epoch 222
Validation binary_cross_entropy = 0.654222
Epoch 223
Validation binary_cross_entropy = 0.642034
Epoch 224
Loss = 1.6131e-05, PNorm = 93.8510, GNorm = 0.0595, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.641713
Epoch 225
Validation binary_cross_entropy = 0.724099
Epoch 226
Validation binary_cross_entropy = 0.790592
Epoch 227
Validation binary_cross_entropy = 0.827664
Epoch 228
Validation binary_cross_entropy = 0.778986
Epoch 229
Loss = 2.5102e-05, PNorm = 93.8983, GNorm = 0.1407, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.753073
Epoch 230
Validation binary_cross_entropy = 0.737113
Epoch 231
Validation binary_cross_entropy = 0.718504
Epoch 232
Validation binary_cross_entropy = 0.701474
Epoch 233
Validation binary_cross_entropy = 0.685673
Epoch 234
Loss = 5.9078e-05, PNorm = 93.9481, GNorm = 0.1545, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.673111
Epoch 235
Validation binary_cross_entropy = 0.679322
Epoch 236
Validation binary_cross_entropy = 0.691560
Epoch 237
Validation binary_cross_entropy = 0.690229
Epoch 238
Validation binary_cross_entropy = 0.707514
Epoch 239
Loss = 6.0680e-04, PNorm = 94.0182, GNorm = 1.8058, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.716370
Epoch 240
Validation binary_cross_entropy = 0.724412
Epoch 241
Validation binary_cross_entropy = 0.733634
Epoch 242
Validation binary_cross_entropy = 0.744011
Epoch 243
Validation binary_cross_entropy = 0.748339
Epoch 244
Loss = 6.0843e-05, PNorm = 94.0685, GNorm = 0.3537, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.744229
Epoch 245
Validation binary_cross_entropy = 0.743988
Epoch 246
Validation binary_cross_entropy = 0.749549
Epoch 247
Validation binary_cross_entropy = 0.767341
Epoch 248
Validation binary_cross_entropy = 0.807521
Epoch 249
Loss = 4.5915e-04, PNorm = 94.1043, GNorm = 0.5134, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.807151
Epoch 250
Validation binary_cross_entropy = 0.784207
Epoch 251
Validation binary_cross_entropy = 0.749150
Epoch 252
Validation binary_cross_entropy = 0.720279
Epoch 253
Validation binary_cross_entropy = 0.704373
Epoch 254
Loss = 7.6314e-05, PNorm = 94.1468, GNorm = 0.0188, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.703862
Epoch 255
Validation binary_cross_entropy = 0.708227
Epoch 256
Validation binary_cross_entropy = 0.718774
Epoch 257
Validation binary_cross_entropy = 0.734657
Epoch 258
Validation binary_cross_entropy = 0.749611
Epoch 259
Loss = 3.4737e-04, PNorm = 94.1779, GNorm = 1.3292, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.766775
Epoch 260
Validation binary_cross_entropy = 0.799316
Epoch 261
Validation binary_cross_entropy = 0.821275
Epoch 262
Validation binary_cross_entropy = 0.765483
Epoch 263
Validation binary_cross_entropy = 0.747589
Epoch 264
Loss = 2.2572e-05, PNorm = 94.2040, GNorm = 0.0866, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.749883
Epoch 265
Validation binary_cross_entropy = 0.766797
Epoch 266
Validation binary_cross_entropy = 0.780647
Epoch 267
Validation binary_cross_entropy = 0.768022
Epoch 268
Validation binary_cross_entropy = 0.767539
Epoch 269
Loss = 4.0322e-05, PNorm = 94.2502, GNorm = 0.2605, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.785995
Epoch 270
Validation binary_cross_entropy = 0.779328
Epoch 271
Validation binary_cross_entropy = 0.740554
Epoch 272
Validation binary_cross_entropy = 0.733662
Epoch 273
Validation binary_cross_entropy = 0.735661
Epoch 274
Loss = 1.9546e-05, PNorm = 94.3263, GNorm = 0.0424, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.739009
Epoch 275
Validation binary_cross_entropy = 0.743934
Epoch 276
Validation binary_cross_entropy = 0.764474
Epoch 277
Validation binary_cross_entropy = 0.799347
Epoch 278
Validation binary_cross_entropy = 0.872130
Epoch 279
Loss = 5.8242e-04, PNorm = 94.3791, GNorm = 2.5220, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.869778
Epoch 280
Validation binary_cross_entropy = 0.788912
Epoch 281
Validation binary_cross_entropy = 0.749063
Epoch 282
Validation binary_cross_entropy = 0.739958
Epoch 283
Validation binary_cross_entropy = 0.740614
Epoch 284
Loss = 8.0072e-04, PNorm = 94.4465, GNorm = 2.0895, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.779119
Epoch 285
Validation binary_cross_entropy = 0.900100
Epoch 286
Validation binary_cross_entropy = 0.922960
Epoch 287
Validation binary_cross_entropy = 0.823896
Epoch 288
Validation binary_cross_entropy = 0.803309
Epoch 289
Loss = 7.4142e-04, PNorm = 94.5575, GNorm = 1.3766, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.806520
Epoch 290
Validation binary_cross_entropy = 0.804430
Epoch 291
Validation binary_cross_entropy = 0.871491
Epoch 292
Validation binary_cross_entropy = 1.011621
Epoch 293
Validation binary_cross_entropy = 1.089344
Epoch 294
Loss = 4.1048e-04, PNorm = 94.6864, GNorm = 0.8691, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.034922
Epoch 295
Validation binary_cross_entropy = 0.865053
Epoch 296
Validation binary_cross_entropy = 0.702100
Epoch 297
Validation binary_cross_entropy = 0.637950
Epoch 298
Validation binary_cross_entropy = 0.657236
Epoch 299
Loss = 1.4178e-04, PNorm = 94.8625, GNorm = 0.2785, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.706464
Model 0 best validation binary_cross_entropy = 0.276116 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.245410
Ensemble test binary_cross_entropy = 0.245410
Fold 4
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_4',
 'save_smiles_splits': False,
 'seed': 4,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.292320
Best score so far is 0.292320 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.476380
Epoch 2
Validation binary_cross_entropy = 0.531591
Epoch 3
Validation binary_cross_entropy = 0.419590
Epoch 4
Loss = 1.2464e-02, PNorm = 87.7270, GNorm = 6.1011, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.427496
Epoch 5
Validation binary_cross_entropy = 0.626789
Epoch 6
Validation binary_cross_entropy = 0.349259
Epoch 7
Validation binary_cross_entropy = 0.517950
Epoch 8
Validation binary_cross_entropy = 0.356871
Epoch 9
Loss = 5.9444e-03, PNorm = 88.0381, GNorm = 1.7984, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.583024
Epoch 10
Validation binary_cross_entropy = 0.337777
Epoch 11
Validation binary_cross_entropy = 0.519096
Epoch 12
Validation binary_cross_entropy = 0.416250
Epoch 13
Validation binary_cross_entropy = 0.418257
Epoch 14
Loss = 3.2633e-03, PNorm = 88.3774, GNorm = 1.0048, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.477410
Epoch 15
Validation binary_cross_entropy = 0.479031
Epoch 16
Validation binary_cross_entropy = 0.387267
Epoch 17
Validation binary_cross_entropy = 0.495117
Epoch 18
Validation binary_cross_entropy = 0.432862
Epoch 19
Loss = 5.6569e-03, PNorm = 88.6443, GNorm = 1.3042, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.468755
Epoch 20
Validation binary_cross_entropy = 0.572527
Epoch 21
Validation binary_cross_entropy = 0.409145
Epoch 22
Validation binary_cross_entropy = 0.422723
Epoch 23
Validation binary_cross_entropy = 0.431050
Epoch 24
Loss = 3.6883e-03, PNorm = 88.8633, GNorm = 1.2187, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.394078
Epoch 25
Validation binary_cross_entropy = 0.398649
Epoch 26
Validation binary_cross_entropy = 0.416069
Epoch 27
Validation binary_cross_entropy = 0.371326
Epoch 28
Validation binary_cross_entropy = 0.371067
Epoch 29
Loss = 2.7098e-03, PNorm = 89.0673, GNorm = 0.8619, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.408356
Epoch 30
Validation binary_cross_entropy = 0.431329
Epoch 31
Validation binary_cross_entropy = 0.429587
Epoch 32
Validation binary_cross_entropy = 0.446659
Epoch 33
Validation binary_cross_entropy = 0.512211
Epoch 34
Loss = 5.4098e-03, PNorm = 89.2689, GNorm = 3.0478, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.512877
Epoch 35
Validation binary_cross_entropy = 0.528432
Epoch 36
Validation binary_cross_entropy = 0.688119
Epoch 37
Validation binary_cross_entropy = 0.497257
Epoch 38
Validation binary_cross_entropy = 0.464563
Epoch 39
Loss = 2.1608e-03, PNorm = 89.4737, GNorm = 0.9302, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.656527
Epoch 40
Validation binary_cross_entropy = 0.498657
Epoch 41
Validation binary_cross_entropy = 0.536369
Epoch 42
Validation binary_cross_entropy = 0.506250
Epoch 43
Validation binary_cross_entropy = 0.469153
Epoch 44
Loss = 2.0912e-03, PNorm = 89.6903, GNorm = 1.9060, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.440885
Epoch 45
Validation binary_cross_entropy = 0.434420
Epoch 46
Validation binary_cross_entropy = 0.414001
Epoch 47
Validation binary_cross_entropy = 0.398629
Epoch 48
Validation binary_cross_entropy = 0.403709
Epoch 49
Loss = 2.9256e-03, PNorm = 89.8594, GNorm = 0.6356, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.391637
Epoch 50
Validation binary_cross_entropy = 0.420433
Epoch 51
Validation binary_cross_entropy = 0.446095
Epoch 52
Validation binary_cross_entropy = 0.477829
Epoch 53
Validation binary_cross_entropy = 0.531819
Epoch 54
Loss = 1.4222e-03, PNorm = 90.0218, GNorm = 0.7214, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.522269
Epoch 55
Validation binary_cross_entropy = 0.497389
Epoch 56
Validation binary_cross_entropy = 0.499429
Epoch 57
Validation binary_cross_entropy = 0.491612
Epoch 58
Validation binary_cross_entropy = 0.491943
Epoch 59
Loss = 2.0572e-03, PNorm = 90.1658, GNorm = 2.3044, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.458383
Epoch 60
Validation binary_cross_entropy = 0.459987
Epoch 61
Validation binary_cross_entropy = 0.492507
Epoch 62
Validation binary_cross_entropy = 0.518095
Epoch 63
Validation binary_cross_entropy = 0.508713
Epoch 64
Loss = 1.3520e-03, PNorm = 90.2930, GNorm = 0.4643, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.526055
Epoch 65
Validation binary_cross_entropy = 0.540326
Epoch 66
Validation binary_cross_entropy = 0.554129
Epoch 67
Validation binary_cross_entropy = 0.567124
Epoch 68
Validation binary_cross_entropy = 0.569731
Epoch 69
Loss = 3.5746e-04, PNorm = 90.4101, GNorm = 0.4518, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.568713
Epoch 70
Validation binary_cross_entropy = 0.591500
Epoch 71
Validation binary_cross_entropy = 0.630784
Epoch 72
Validation binary_cross_entropy = 0.563569
Epoch 73
Validation binary_cross_entropy = 0.569027
Epoch 74
Loss = 1.0386e-03, PNorm = 90.5343, GNorm = 0.3983, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.557163
Epoch 75
Validation binary_cross_entropy = 0.611320
Epoch 76
Validation binary_cross_entropy = 0.667221
Epoch 77
Validation binary_cross_entropy = 0.562432
Epoch 78
Validation binary_cross_entropy = 0.575047
Epoch 79
Loss = 2.4281e-03, PNorm = 90.7031, GNorm = 1.5095, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.565183
Epoch 80
Validation binary_cross_entropy = 0.652244
Epoch 81
Validation binary_cross_entropy = 0.642974
Epoch 82
Validation binary_cross_entropy = 0.618830
Epoch 83
Validation binary_cross_entropy = 0.666582
Epoch 84
Loss = 1.6822e-03, PNorm = 90.9023, GNorm = 1.5501, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.630801
Epoch 85
Validation binary_cross_entropy = 0.652976
Epoch 86
Validation binary_cross_entropy = 0.662604
Epoch 87
Validation binary_cross_entropy = 0.639798
Epoch 88
Validation binary_cross_entropy = 0.628417
Epoch 89
Loss = 1.5247e-03, PNorm = 91.1482, GNorm = 1.8166, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.643392
Epoch 90
Validation binary_cross_entropy = 0.675795
Epoch 91
Validation binary_cross_entropy = 0.690542
Epoch 92
Validation binary_cross_entropy = 0.654133
Epoch 93
Validation binary_cross_entropy = 0.642803
Epoch 94
Loss = 1.2210e-03, PNorm = 91.3130, GNorm = 1.4375, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.631248
Epoch 95
Validation binary_cross_entropy = 0.631388
Epoch 96
Validation binary_cross_entropy = 0.601360
Epoch 97
Validation binary_cross_entropy = 0.590492
Epoch 98
Validation binary_cross_entropy = 0.587865
Epoch 99
Loss = 4.8393e-04, PNorm = 91.4303, GNorm = 0.2494, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.588892
Epoch 100
Validation binary_cross_entropy = 0.609925
Epoch 101
Validation binary_cross_entropy = 0.645424
Epoch 102
Validation binary_cross_entropy = 0.679645
Epoch 103
Validation binary_cross_entropy = 0.713434
Epoch 104
Loss = 5.5127e-04, PNorm = 91.5664, GNorm = 1.4577, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.720028
Epoch 105
Validation binary_cross_entropy = 0.700676
Epoch 106
Validation binary_cross_entropy = 0.761300
Epoch 107
Validation binary_cross_entropy = 0.703045
Epoch 108
Validation binary_cross_entropy = 0.790975
Epoch 109
Loss = 1.6593e-03, PNorm = 91.7810, GNorm = 3.0350, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.778388
Epoch 110
Validation binary_cross_entropy = 0.656640
Epoch 111
Validation binary_cross_entropy = 0.609728
Epoch 112
Validation binary_cross_entropy = 0.576280
Epoch 113
Validation binary_cross_entropy = 0.580051
Epoch 114
Loss = 1.4125e-03, PNorm = 92.0179, GNorm = 1.3622, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.654586
Epoch 115
Validation binary_cross_entropy = 0.684893
Epoch 116
Validation binary_cross_entropy = 0.646616
Epoch 117
Validation binary_cross_entropy = 0.625504
Epoch 118
Validation binary_cross_entropy = 0.613246
Epoch 119
Loss = 1.0850e-03, PNorm = 92.2116, GNorm = 0.7641, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.637681
Epoch 120
Validation binary_cross_entropy = 0.651745
Epoch 121
Validation binary_cross_entropy = 0.617913
Epoch 122
Validation binary_cross_entropy = 0.611897
Epoch 123
Validation binary_cross_entropy = 0.620812
Epoch 124
Loss = 6.4995e-04, PNorm = 92.3614, GNorm = 0.1697, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.639250
Epoch 125
Validation binary_cross_entropy = 0.674685
Epoch 126
Validation binary_cross_entropy = 0.725360
Epoch 127
Validation binary_cross_entropy = 0.734962
Epoch 128
Validation binary_cross_entropy = 0.726818
Epoch 129
Loss = 1.2342e-04, PNorm = 92.4756, GNorm = 0.1392, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.732498
Epoch 130
Validation binary_cross_entropy = 0.741737
Epoch 131
Validation binary_cross_entropy = 0.763226
Epoch 132
Validation binary_cross_entropy = 0.806696
Epoch 133
Validation binary_cross_entropy = 0.845850
Epoch 134
Loss = 7.8487e-04, PNorm = 92.5980, GNorm = 0.7388, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.875797
Epoch 135
Validation binary_cross_entropy = 0.840958
Epoch 136
Validation binary_cross_entropy = 0.806225
Epoch 137
Validation binary_cross_entropy = 0.800474
Epoch 138
Validation binary_cross_entropy = 0.813943
Epoch 139
Loss = 7.8726e-04, PNorm = 92.7483, GNorm = 0.1015, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.824057
Epoch 140
Validation binary_cross_entropy = 0.842595
Epoch 141
Validation binary_cross_entropy = 0.858937
Epoch 142
Validation binary_cross_entropy = 0.860959
Epoch 143
Validation binary_cross_entropy = 0.787432
Epoch 144
Loss = 3.3030e-05, PNorm = 92.8662, GNorm = 0.0427, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.733858
Epoch 145
Validation binary_cross_entropy = 0.743033
Epoch 146
Validation binary_cross_entropy = 0.747715
Epoch 147
Validation binary_cross_entropy = 0.776717
Epoch 148
Validation binary_cross_entropy = 0.792250
Epoch 149
Loss = 4.1171e-05, PNorm = 93.0519, GNorm = 0.0579, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.820511
Epoch 150
Validation binary_cross_entropy = 0.687983
Epoch 151
Validation binary_cross_entropy = 0.668686
Epoch 152
Validation binary_cross_entropy = 0.672024
Epoch 153
Validation binary_cross_entropy = 0.691573
Epoch 154
Loss = 1.7297e-03, PNorm = 93.1963, GNorm = 2.1765, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.723327
Epoch 155
Validation binary_cross_entropy = 0.761841
Epoch 156
Validation binary_cross_entropy = 0.778479
Epoch 157
Validation binary_cross_entropy = 0.754005
Epoch 158
Validation binary_cross_entropy = 0.723161
Epoch 159
Loss = 5.9228e-04, PNorm = 93.3503, GNorm = 1.1389, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.713626
Epoch 160
Validation binary_cross_entropy = 0.707265
Epoch 161
Validation binary_cross_entropy = 0.691532
Epoch 162
Validation binary_cross_entropy = 0.676984
Epoch 163
Validation binary_cross_entropy = 0.670817
Epoch 164
Loss = 6.1908e-04, PNorm = 93.4796, GNorm = 1.2865, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.684068
Epoch 165
Validation binary_cross_entropy = 0.716890
Epoch 166
Validation binary_cross_entropy = 0.742854
Epoch 167
Validation binary_cross_entropy = 0.743541
Epoch 168
Validation binary_cross_entropy = 0.762270
Epoch 169
Loss = 1.2253e-04, PNorm = 93.5970, GNorm = 0.5028, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.776762
Epoch 170
Validation binary_cross_entropy = 0.783049
Epoch 171
Validation binary_cross_entropy = 0.788080
Epoch 172
Validation binary_cross_entropy = 0.788553
Epoch 173
Validation binary_cross_entropy = 0.789515
Epoch 174
Loss = 9.3550e-05, PNorm = 93.6899, GNorm = 0.1910, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.793432
Epoch 175
Validation binary_cross_entropy = 0.794012
Epoch 176
Validation binary_cross_entropy = 0.793413
Epoch 177
Validation binary_cross_entropy = 0.801954
Epoch 178
Validation binary_cross_entropy = 0.825831
Epoch 179
Loss = 2.1622e-04, PNorm = 93.7541, GNorm = 0.0200, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.883804
Epoch 180
Validation binary_cross_entropy = 0.872787
Epoch 181
Validation binary_cross_entropy = 0.834684
Epoch 182
Validation binary_cross_entropy = 0.825673
Epoch 183
Validation binary_cross_entropy = 0.818531
Epoch 184
Loss = 9.9300e-05, PNorm = 93.8045, GNorm = 0.1477, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.819728
Epoch 185
Validation binary_cross_entropy = 0.816061
Epoch 186
Validation binary_cross_entropy = 0.816324
Epoch 187
Validation binary_cross_entropy = 0.778351
Epoch 188
Validation binary_cross_entropy = 0.767964
Epoch 189
Loss = 8.0804e-04, PNorm = 93.8416, GNorm = 0.0861, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.762039
Epoch 190
Validation binary_cross_entropy = 0.764808
Epoch 191
Validation binary_cross_entropy = 0.774141
Epoch 192
Validation binary_cross_entropy = 0.793401
Epoch 193
Validation binary_cross_entropy = 0.811053
Epoch 194
Loss = 9.9078e-04, PNorm = 93.8767, GNorm = 0.6400, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.819002
Epoch 195
Validation binary_cross_entropy = 0.791389
Epoch 196
Validation binary_cross_entropy = 0.785979
Epoch 197
Validation binary_cross_entropy = 0.786634
Epoch 198
Validation binary_cross_entropy = 0.789405
Epoch 199
Loss = 2.6188e-05, PNorm = 93.9222, GNorm = 0.1114, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.795749
Epoch 200
Validation binary_cross_entropy = 0.805054
Epoch 201
Validation binary_cross_entropy = 0.814448
Epoch 202
Validation binary_cross_entropy = 0.815134
Epoch 203
Validation binary_cross_entropy = 0.817444
Epoch 204
Loss = 1.2739e-05, PNorm = 93.9741, GNorm = 0.0459, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.818960
Epoch 205
Validation binary_cross_entropy = 0.822440
Epoch 206
Validation binary_cross_entropy = 0.827437
Epoch 207
Validation binary_cross_entropy = 0.832935
Epoch 208
Validation binary_cross_entropy = 0.837247
Epoch 209
Loss = 2.8723e-04, PNorm = 94.0101, GNorm = 0.0132, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.855597
Epoch 210
Validation binary_cross_entropy = 0.878310
Epoch 211
Validation binary_cross_entropy = 0.875936
Epoch 212
Validation binary_cross_entropy = 0.857951
Epoch 213
Validation binary_cross_entropy = 0.845805
Epoch 214
Loss = 3.9553e-05, PNorm = 94.0389, GNorm = 0.1069, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.843320
Epoch 215
Validation binary_cross_entropy = 0.845612
Epoch 216
Validation binary_cross_entropy = 0.850221
Epoch 217
Validation binary_cross_entropy = 0.863632
Epoch 218
Validation binary_cross_entropy = 0.875085
Epoch 219
Loss = 1.1805e-04, PNorm = 94.0594, GNorm = 0.5148, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.885097
Epoch 220
Validation binary_cross_entropy = 0.892633
Epoch 221
Validation binary_cross_entropy = 0.894058
Epoch 222
Validation binary_cross_entropy = 0.923204
Epoch 223
Validation binary_cross_entropy = 0.964984
Epoch 224
Loss = 1.3227e-04, PNorm = 94.0874, GNorm = 0.3941, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.971646
Epoch 225
Validation binary_cross_entropy = 0.930209
Epoch 226
Validation binary_cross_entropy = 0.891653
Epoch 227
Validation binary_cross_entropy = 0.860839
Epoch 228
Validation binary_cross_entropy = 0.843397
Epoch 229
Loss = 7.1984e-05, PNorm = 94.1222, GNorm = 0.3093, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.833108
Epoch 230
Validation binary_cross_entropy = 0.824539
Epoch 231
Validation binary_cross_entropy = 0.819923
Epoch 232
Validation binary_cross_entropy = 0.815557
Epoch 233
Validation binary_cross_entropy = 0.817833
Epoch 234
Loss = 1.4403e-04, PNorm = 94.1514, GNorm = 0.8690, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.843556
Epoch 235
Validation binary_cross_entropy = 0.881248
Epoch 236
Validation binary_cross_entropy = 0.893151
Epoch 237
Validation binary_cross_entropy = 0.894267
Epoch 238
Validation binary_cross_entropy = 0.903606
Epoch 239
Loss = 1.3255e-05, PNorm = 94.1866, GNorm = 0.0722, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.915037
Epoch 240
Validation binary_cross_entropy = 0.927308
Epoch 241
Validation binary_cross_entropy = 0.938148
Epoch 242
Validation binary_cross_entropy = 0.947172
Epoch 243
Validation binary_cross_entropy = 0.954986
Epoch 244
Loss = 1.8798e-05, PNorm = 94.2104, GNorm = 0.0080, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.963617
Epoch 245
Validation binary_cross_entropy = 0.953639
Epoch 246
Validation binary_cross_entropy = 0.931067
Epoch 247
Validation binary_cross_entropy = 0.915176
Epoch 248
Validation binary_cross_entropy = 0.902261
Epoch 249
Loss = 2.9544e-05, PNorm = 94.2328, GNorm = 0.0279, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.894637
Epoch 250
Validation binary_cross_entropy = 0.889367
Epoch 251
Validation binary_cross_entropy = 0.885219
Epoch 252
Validation binary_cross_entropy = 0.881928
Epoch 253
Validation binary_cross_entropy = 0.880334
Epoch 254
Loss = 1.3702e-05, PNorm = 94.2512, GNorm = 0.0174, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.878302
Epoch 255
Validation binary_cross_entropy = 0.876325
Epoch 256
Validation binary_cross_entropy = 0.874242
Epoch 257
Validation binary_cross_entropy = 0.871500
Epoch 258
Validation binary_cross_entropy = 0.869746
Epoch 259
Loss = 1.9224e-05, PNorm = 94.2657, GNorm = 0.0052, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.870428
Epoch 260
Validation binary_cross_entropy = 0.874806
Epoch 261
Validation binary_cross_entropy = 0.882284
Epoch 262
Validation binary_cross_entropy = 0.890713
Epoch 263
Validation binary_cross_entropy = 0.941410
Epoch 264
Loss = 1.9156e-04, PNorm = 94.2798, GNorm = 0.2094, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.959022
Epoch 265
Validation binary_cross_entropy = 0.964196
Epoch 266
Validation binary_cross_entropy = 0.964571
Epoch 267
Validation binary_cross_entropy = 0.939704
Epoch 268
Validation binary_cross_entropy = 0.927422
Epoch 269
Loss = 4.0226e-06, PNorm = 94.3200, GNorm = 0.0029, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.925435
Epoch 270
Validation binary_cross_entropy = 0.913231
Epoch 271
Validation binary_cross_entropy = 0.914116
Epoch 272
Validation binary_cross_entropy = 0.926034
Epoch 273
Validation binary_cross_entropy = 0.918354
Epoch 274
Loss = 2.6553e-05, PNorm = 94.3577, GNorm = 0.0205, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.908425
Epoch 275
Validation binary_cross_entropy = 0.888112
Epoch 276
Validation binary_cross_entropy = 0.865813
Epoch 277
Validation binary_cross_entropy = 0.855456
Epoch 278
Validation binary_cross_entropy = 0.853566
Epoch 279
Loss = 1.7197e-05, PNorm = 94.3876, GNorm = 0.1287, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.854453
Epoch 280
Validation binary_cross_entropy = 0.863199
Epoch 281
Validation binary_cross_entropy = 0.868644
Epoch 282
Validation binary_cross_entropy = 0.880680
Epoch 283
Validation binary_cross_entropy = 0.894574
Epoch 284
Loss = 1.4387e-04, PNorm = 94.4022, GNorm = 0.9269, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.913759
Epoch 285
Validation binary_cross_entropy = 0.940423
Epoch 286
Validation binary_cross_entropy = 0.962739
Epoch 287
Validation binary_cross_entropy = 0.966697
Epoch 288
Validation binary_cross_entropy = 0.957302
Epoch 289
Loss = 1.0292e-05, PNorm = 94.4229, GNorm = 0.0391, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.954794
Epoch 290
Validation binary_cross_entropy = 0.954226
Epoch 291
Validation binary_cross_entropy = 0.974770
Epoch 292
Validation binary_cross_entropy = 1.015192
Epoch 293
Validation binary_cross_entropy = 1.060387
Epoch 294
Loss = 1.8760e-05, PNorm = 94.4614, GNorm = 0.0168, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.101323
Epoch 295
Validation binary_cross_entropy = 1.116238
Epoch 296
Validation binary_cross_entropy = 1.046532
Epoch 297
Validation binary_cross_entropy = 1.010009
Epoch 298
Validation binary_cross_entropy = 1.039231
Epoch 299
Loss = 1.9554e-03, PNorm = 94.5307, GNorm = 3.1514, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.962856
Model 0 best validation binary_cross_entropy = 0.292320 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.254593
Ensemble test binary_cross_entropy = 0.254593
Fold 5
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_5',
 'save_smiles_splits': False,
 'seed': 5,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.813881
Best score so far is 0.813881 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.298206
Best score so far is 0.298206 at epoch 1 
Epoch 2
Validation binary_cross_entropy = 0.624998
Epoch 3
Validation binary_cross_entropy = 0.317518
Epoch 4
Loss = 9.1879e-03, PNorm = 87.7228, GNorm = 1.8561, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.433773
Epoch 5
Validation binary_cross_entropy = 0.495498
Epoch 6
Validation binary_cross_entropy = 0.303229
Epoch 7
Validation binary_cross_entropy = 0.602929
Epoch 8
Validation binary_cross_entropy = 0.311324
Epoch 9
Loss = 7.2415e-03, PNorm = 88.0205, GNorm = 1.4850, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.399639
Epoch 10
Validation binary_cross_entropy = 0.353954
Epoch 11
Validation binary_cross_entropy = 0.359711
Epoch 12
Validation binary_cross_entropy = 0.386360
Epoch 13
Validation binary_cross_entropy = 0.343494
Epoch 14
Loss = 5.8462e-03, PNorm = 88.3454, GNorm = 1.6921, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.363987
Epoch 15
Validation binary_cross_entropy = 0.368959
Epoch 16
Validation binary_cross_entropy = 0.382696
Epoch 17
Validation binary_cross_entropy = 0.359278
Epoch 18
Validation binary_cross_entropy = 0.375833
Epoch 19
Loss = 5.4033e-03, PNorm = 88.6212, GNorm = 2.3236, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.440132
Epoch 20
Validation binary_cross_entropy = 0.452086
Epoch 21
Validation binary_cross_entropy = 0.427266
Epoch 22
Validation binary_cross_entropy = 0.408584
Epoch 23
Validation binary_cross_entropy = 0.467514
Epoch 24
Loss = 4.2532e-03, PNorm = 88.8415, GNorm = 2.4127, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.420076
Epoch 25
Validation binary_cross_entropy = 0.525735
Epoch 26
Validation binary_cross_entropy = 0.754000
Epoch 27
Validation binary_cross_entropy = 0.486007
Epoch 28
Validation binary_cross_entropy = 0.451365
Epoch 29
Loss = 3.3537e-03, PNorm = 89.0384, GNorm = 4.4092, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.454572
Epoch 30
Validation binary_cross_entropy = 0.451190
Epoch 31
Validation binary_cross_entropy = 0.448721
Epoch 32
Validation binary_cross_entropy = 0.518822
Epoch 33
Validation binary_cross_entropy = 0.455686
Epoch 34
Loss = 2.2029e-03, PNorm = 89.2292, GNorm = 2.2819, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.459076
Epoch 35
Validation binary_cross_entropy = 0.446928
Epoch 36
Validation binary_cross_entropy = 0.445528
Epoch 37
Validation binary_cross_entropy = 0.449800
Epoch 38
Validation binary_cross_entropy = 0.491245
Epoch 39
Loss = 3.2400e-03, PNorm = 89.4345, GNorm = 0.7641, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.410821
Epoch 40
Validation binary_cross_entropy = 0.423307
Epoch 41
Validation binary_cross_entropy = 0.434453
Epoch 42
Validation binary_cross_entropy = 0.471271
Epoch 43
Validation binary_cross_entropy = 0.496991
Epoch 44
Loss = 3.1531e-03, PNorm = 89.6061, GNorm = 1.1217, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.567232
Epoch 45
Validation binary_cross_entropy = 0.480122
Epoch 46
Validation binary_cross_entropy = 0.463694
Epoch 47
Validation binary_cross_entropy = 0.508578
Epoch 48
Validation binary_cross_entropy = 0.459959
Epoch 49
Loss = 1.0434e-03, PNorm = 89.7403, GNorm = 1.4509, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.487852
Epoch 50
Validation binary_cross_entropy = 0.500518
Epoch 51
Validation binary_cross_entropy = 0.508723
Epoch 52
Validation binary_cross_entropy = 0.507090
Epoch 53
Validation binary_cross_entropy = 0.484660
Epoch 54
Loss = 9.3500e-04, PNorm = 89.8792, GNorm = 0.5922, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.470598
Epoch 55
Validation binary_cross_entropy = 0.477901
Epoch 56
Validation binary_cross_entropy = 0.488881
Epoch 57
Validation binary_cross_entropy = 0.517536
Epoch 58
Validation binary_cross_entropy = 0.531618
Epoch 59
Loss = 2.3823e-04, PNorm = 90.0087, GNorm = 0.6178, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.555885
Epoch 60
Validation binary_cross_entropy = 0.569112
Epoch 61
Validation binary_cross_entropy = 0.607199
Epoch 62
Validation binary_cross_entropy = 0.582358
Epoch 63
Validation binary_cross_entropy = 0.538107
Epoch 64
Loss = 1.1482e-03, PNorm = 90.1404, GNorm = 1.0630, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.539702
Epoch 65
Validation binary_cross_entropy = 0.522200
Epoch 66
Validation binary_cross_entropy = 0.565826
Epoch 67
Validation binary_cross_entropy = 0.591046
Epoch 68
Validation binary_cross_entropy = 0.633425
Epoch 69
Loss = 3.3413e-03, PNorm = 90.3059, GNorm = 2.0942, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.589821
Epoch 70
Validation binary_cross_entropy = 0.615104
Epoch 71
Validation binary_cross_entropy = 0.586519
Epoch 72
Validation binary_cross_entropy = 0.597616
Epoch 73
Validation binary_cross_entropy = 0.621681
Epoch 74
Loss = 2.6873e-03, PNorm = 90.4829, GNorm = 1.9582, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.664247
Epoch 75
Validation binary_cross_entropy = 0.630094
Epoch 76
Validation binary_cross_entropy = 0.648916
Epoch 77
Validation binary_cross_entropy = 0.703311
Epoch 78
Validation binary_cross_entropy = 0.704248
Epoch 79
Loss = 1.7415e-04, PNorm = 90.6701, GNorm = 0.2306, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.689594
Epoch 80
Validation binary_cross_entropy = 0.665580
Epoch 81
Validation binary_cross_entropy = 0.620701
Epoch 82
Validation binary_cross_entropy = 0.603457
Epoch 83
Validation binary_cross_entropy = 0.765898
Epoch 84
Loss = 3.5276e-03, PNorm = 90.8135, GNorm = 5.8999, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.601435
Epoch 85
Validation binary_cross_entropy = 0.657416
Epoch 86
Validation binary_cross_entropy = 0.635021
Epoch 87
Validation binary_cross_entropy = 0.780184
Epoch 88
Validation binary_cross_entropy = 0.711278
Epoch 89
Loss = 1.8362e-03, PNorm = 91.0752, GNorm = 1.6922, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.635987
Epoch 90
Validation binary_cross_entropy = 0.708791
Epoch 91
Validation binary_cross_entropy = 0.671133
Epoch 92
Validation binary_cross_entropy = 0.683510
Epoch 93
Validation binary_cross_entropy = 0.694810
Epoch 94
Loss = 8.0218e-04, PNorm = 91.3214, GNorm = 0.9901, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.700210
Epoch 95
Validation binary_cross_entropy = 0.703765
Epoch 96
Validation binary_cross_entropy = 1.032683
Epoch 97
Validation binary_cross_entropy = 0.682846
Epoch 98
Validation binary_cross_entropy = 0.702919
Epoch 99
Loss = 3.0180e-03, PNorm = 91.5917, GNorm = 0.2184, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.601698
Epoch 100
Validation binary_cross_entropy = 0.544752
Epoch 101
Validation binary_cross_entropy = 0.536995
Epoch 102
Validation binary_cross_entropy = 0.511477
Epoch 103
Validation binary_cross_entropy = 0.507887
Epoch 104
Loss = 6.8747e-04, PNorm = 91.9005, GNorm = 0.8870, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.535631
Epoch 105
Validation binary_cross_entropy = 0.550047
Epoch 106
Validation binary_cross_entropy = 0.542878
Epoch 107
Validation binary_cross_entropy = 0.529329
Epoch 108
Validation binary_cross_entropy = 0.530916
Epoch 109
Loss = 5.8715e-04, PNorm = 92.1254, GNorm = 0.1992, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.540737
Epoch 110
Validation binary_cross_entropy = 0.557946
Epoch 111
Validation binary_cross_entropy = 0.579337
Epoch 112
Validation binary_cross_entropy = 0.598866
Epoch 113
Validation binary_cross_entropy = 0.577831
Epoch 114
Loss = 1.0508e-03, PNorm = 92.2763, GNorm = 0.2102, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.552102
Epoch 115
Validation binary_cross_entropy = 0.553729
Epoch 116
Validation binary_cross_entropy = 0.559099
Epoch 117
Validation binary_cross_entropy = 0.571218
Epoch 118
Validation binary_cross_entropy = 0.586251
Epoch 119
Loss = 1.5619e-03, PNorm = 92.4045, GNorm = 1.8074, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.583876
Epoch 120
Validation binary_cross_entropy = 0.609775
Epoch 121
Validation binary_cross_entropy = 0.743936
Epoch 122
Validation binary_cross_entropy = 0.666398
Epoch 123
Validation binary_cross_entropy = 0.590622
Epoch 124
Loss = 1.2581e-04, PNorm = 92.5296, GNorm = 0.0808, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.566948
Epoch 125
Validation binary_cross_entropy = 0.565308
Epoch 126
Validation binary_cross_entropy = 0.572728
Epoch 127
Validation binary_cross_entropy = 0.604492
Epoch 128
Validation binary_cross_entropy = 0.609754
Epoch 129
Loss = 6.2943e-04, PNorm = 92.6413, GNorm = 0.7294, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.587931
Epoch 130
Validation binary_cross_entropy = 0.585778
Epoch 131
Validation binary_cross_entropy = 0.580956
Epoch 132
Validation binary_cross_entropy = 0.590521
Epoch 133
Validation binary_cross_entropy = 0.600181
Epoch 134
Loss = 1.1132e-03, PNorm = 92.7521, GNorm = 1.8276, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.596559
Epoch 135
Validation binary_cross_entropy = 0.625592
Epoch 136
Validation binary_cross_entropy = 0.631948
Epoch 137
Validation binary_cross_entropy = 0.617712
Epoch 138
Validation binary_cross_entropy = 0.606461
Epoch 139
Loss = 2.5659e-04, PNorm = 92.8486, GNorm = 0.7540, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.596064
Epoch 140
Validation binary_cross_entropy = 0.587709
Epoch 141
Validation binary_cross_entropy = 0.574820
Epoch 142
Validation binary_cross_entropy = 0.551372
Epoch 143
Validation binary_cross_entropy = 0.562344
Epoch 144
Loss = 3.7014e-04, PNorm = 92.9442, GNorm = 0.2905, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.594953
Epoch 145
Validation binary_cross_entropy = 0.621361
Epoch 146
Validation binary_cross_entropy = 0.613995
Epoch 147
Validation binary_cross_entropy = 0.599216
Epoch 148
Validation binary_cross_entropy = 0.603363
Epoch 149
Loss = 8.2785e-04, PNorm = 93.0518, GNorm = 1.6233, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.607259
Epoch 150
Validation binary_cross_entropy = 0.609783
Epoch 151
Validation binary_cross_entropy = 0.622921
Epoch 152
Validation binary_cross_entropy = 0.632875
Epoch 153
Validation binary_cross_entropy = 0.635104
Epoch 154
Loss = 5.0473e-05, PNorm = 93.1333, GNorm = 0.0749, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.640808
Epoch 155
Validation binary_cross_entropy = 0.650114
Epoch 156
Validation binary_cross_entropy = 0.668082
Epoch 157
Validation binary_cross_entropy = 0.726343
Epoch 158
Validation binary_cross_entropy = 0.767943
Epoch 159
Loss = 2.6240e-04, PNorm = 93.1833, GNorm = 1.1321, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.767652
Epoch 160
Validation binary_cross_entropy = 0.729248
Epoch 161
Validation binary_cross_entropy = 0.711669
Epoch 162
Validation binary_cross_entropy = 0.701648
Epoch 163
Validation binary_cross_entropy = 0.709481
Epoch 164
Loss = 1.1209e-03, PNorm = 93.2401, GNorm = 0.6641, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.681257
Epoch 165
Validation binary_cross_entropy = 0.687853
Epoch 166
Validation binary_cross_entropy = 0.745911
Epoch 167
Validation binary_cross_entropy = 0.773859
Epoch 168
Validation binary_cross_entropy = 0.766416
Epoch 169
Loss = 2.5430e-04, PNorm = 93.3018, GNorm = 0.0699, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.720233
Epoch 170
Validation binary_cross_entropy = 0.696179
Epoch 171
Validation binary_cross_entropy = 0.691050
Epoch 172
Validation binary_cross_entropy = 0.686874
Epoch 173
Validation binary_cross_entropy = 0.719897
Epoch 174
Loss = 7.0498e-04, PNorm = 93.3603, GNorm = 1.2802, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.703521
Epoch 175
Validation binary_cross_entropy = 0.659878
Epoch 176
Validation binary_cross_entropy = 0.642424
Epoch 177
Validation binary_cross_entropy = 0.659941
Epoch 178
Validation binary_cross_entropy = 0.699718
Epoch 179
Loss = 1.1115e-04, PNorm = 93.4730, GNorm = 0.5695, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.860930
Epoch 180
Validation binary_cross_entropy = 0.993105
Epoch 181
Validation binary_cross_entropy = 0.836739
Epoch 182
Validation binary_cross_entropy = 0.725582
Epoch 183
Validation binary_cross_entropy = 0.683834
Epoch 184
Loss = 1.0659e-03, PNorm = 93.5927, GNorm = 1.2778, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.653841
Epoch 185
Validation binary_cross_entropy = 0.633464
Epoch 186
Validation binary_cross_entropy = 0.638492
Epoch 187
Validation binary_cross_entropy = 0.633070
Epoch 188
Validation binary_cross_entropy = 0.605703
Epoch 189
Loss = 6.9662e-04, PNorm = 93.7567, GNorm = 0.8624, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.562991
Epoch 190
Validation binary_cross_entropy = 0.540832
Epoch 191
Validation binary_cross_entropy = 0.540770
Epoch 192
Validation binary_cross_entropy = 0.545428
Epoch 193
Validation binary_cross_entropy = 0.590399
Epoch 194
Loss = 2.6347e-04, PNorm = 93.8659, GNorm = 0.7889, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.617185
Epoch 195
Validation binary_cross_entropy = 0.619486
Epoch 196
Validation binary_cross_entropy = 0.597877
Epoch 197
Validation binary_cross_entropy = 0.570662
Epoch 198
Validation binary_cross_entropy = 0.565920
Epoch 199
Loss = 3.0012e-04, PNorm = 93.9409, GNorm = 0.9310, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.570439
Epoch 200
Validation binary_cross_entropy = 0.573518
Epoch 201
Validation binary_cross_entropy = 0.578869
Epoch 202
Validation binary_cross_entropy = 0.586394
Epoch 203
Validation binary_cross_entropy = 0.597733
Epoch 204
Loss = 1.2570e-04, PNorm = 93.9936, GNorm = 0.3875, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.612870
Epoch 205
Validation binary_cross_entropy = 0.625293
Epoch 206
Validation binary_cross_entropy = 0.623234
Epoch 207
Validation binary_cross_entropy = 0.620376
Epoch 208
Validation binary_cross_entropy = 0.626317
Epoch 209
Loss = 4.3598e-04, PNorm = 94.0386, GNorm = 2.2308, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.632665
Epoch 210
Validation binary_cross_entropy = 0.656488
Epoch 211
Validation binary_cross_entropy = 0.690676
Epoch 212
Validation binary_cross_entropy = 0.719287
Epoch 213
Validation binary_cross_entropy = 0.756105
Epoch 214
Loss = 2.8001e-04, PNorm = 94.0868, GNorm = 0.1307, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.755183
Epoch 215
Validation binary_cross_entropy = 0.729098
Epoch 216
Validation binary_cross_entropy = 0.693730
Epoch 217
Validation binary_cross_entropy = 0.672057
Epoch 218
Validation binary_cross_entropy = 0.662099
Epoch 219
Loss = 8.5825e-04, PNorm = 94.1382, GNorm = 1.6240, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.680041
Epoch 220
Validation binary_cross_entropy = 0.700273
Epoch 221
Validation binary_cross_entropy = 0.705274
Epoch 222
Validation binary_cross_entropy = 0.707516
Epoch 223
Validation binary_cross_entropy = 0.709736
Epoch 224
Loss = 1.0727e-04, PNorm = 94.1853, GNorm = 0.3734, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.717776
Epoch 225
Validation binary_cross_entropy = 0.727586
Epoch 226
Validation binary_cross_entropy = 0.717173
Epoch 227
Validation binary_cross_entropy = 0.722264
Epoch 228
Validation binary_cross_entropy = 0.745634
Epoch 229
Loss = 2.9364e-04, PNorm = 94.2431, GNorm = 0.0546, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.738770
Epoch 230
Validation binary_cross_entropy = 0.739044
Epoch 231
Validation binary_cross_entropy = 0.742640
Epoch 232
Validation binary_cross_entropy = 0.760748
Epoch 233
Validation binary_cross_entropy = 0.782022
Epoch 234
Loss = 9.3314e-04, PNorm = 94.2951, GNorm = 0.5571, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.817977
Epoch 235
Validation binary_cross_entropy = 0.848367
Epoch 236
Validation binary_cross_entropy = 0.861983
Epoch 237
Validation binary_cross_entropy = 0.877387
Epoch 238
Validation binary_cross_entropy = 0.882732
Epoch 239
Loss = 1.1076e-04, PNorm = 94.3360, GNorm = 0.2952, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.889259
Epoch 240
Validation binary_cross_entropy = 0.891921
Epoch 241
Validation binary_cross_entropy = 0.893307
Epoch 242
Validation binary_cross_entropy = 0.886372
Epoch 243
Validation binary_cross_entropy = 0.873271
Epoch 244
Loss = 3.0583e-05, PNorm = 94.3780, GNorm = 0.3326, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.871843
Epoch 245
Validation binary_cross_entropy = 0.879954
Epoch 246
Validation binary_cross_entropy = 0.890430
Epoch 247
Validation binary_cross_entropy = 0.902046
Epoch 248
Validation binary_cross_entropy = 0.913104
Epoch 249
Loss = 5.4478e-06, PNorm = 94.4226, GNorm = 0.0123, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.923207
Epoch 250
Validation binary_cross_entropy = 0.930174
Epoch 251
Validation binary_cross_entropy = 0.935855
Epoch 252
Validation binary_cross_entropy = 0.937304
Epoch 253
Validation binary_cross_entropy = 0.938952
Epoch 254
Loss = 1.2083e-04, PNorm = 94.4525, GNorm = 0.4328, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.940355
Epoch 255
Validation binary_cross_entropy = 0.946851
Epoch 256
Validation binary_cross_entropy = 0.956393
Epoch 257
Validation binary_cross_entropy = 0.935744
Epoch 258
Validation binary_cross_entropy = 0.911578
Epoch 259
Loss = 9.4306e-05, PNorm = 94.4896, GNorm = 0.6091, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.914889
Epoch 260
Validation binary_cross_entropy = 0.927877
Epoch 261
Validation binary_cross_entropy = 0.936785
Epoch 262
Validation binary_cross_entropy = 0.951221
Epoch 263
Validation binary_cross_entropy = 0.992955
Epoch 264
Loss = 2.9275e-04, PNorm = 94.5173, GNorm = 1.3130, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.004811
Epoch 265
Validation binary_cross_entropy = 0.947998
Epoch 266
Validation binary_cross_entropy = 0.909564
Epoch 267
Validation binary_cross_entropy = 0.888255
Epoch 268
Validation binary_cross_entropy = 0.870547
Epoch 269
Loss = 4.9064e-05, PNorm = 94.5761, GNorm = 0.2565, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.857853
Epoch 270
Validation binary_cross_entropy = 0.853750
Epoch 271
Validation binary_cross_entropy = 0.849382
Epoch 272
Validation binary_cross_entropy = 0.849400
Epoch 273
Validation binary_cross_entropy = 0.854769
Epoch 274
Loss = 4.7912e-05, PNorm = 94.6143, GNorm = 0.4427, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.859491
Epoch 275
Validation binary_cross_entropy = 0.860225
Epoch 276
Validation binary_cross_entropy = 0.870360
Epoch 277
Validation binary_cross_entropy = 0.858266
Epoch 278
Validation binary_cross_entropy = 0.847538
Epoch 279
Loss = 2.9426e-04, PNorm = 94.6578, GNorm = 1.0022, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.855514
Epoch 280
Validation binary_cross_entropy = 0.892546
Epoch 281
Validation binary_cross_entropy = 0.934079
Epoch 282
Validation binary_cross_entropy = 0.965075
Epoch 283
Validation binary_cross_entropy = 0.972698
Epoch 284
Loss = 5.3275e-04, PNorm = 94.7138, GNorm = 0.0390, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.914650
Epoch 285
Validation binary_cross_entropy = 0.894630
Epoch 286
Validation binary_cross_entropy = 0.893713
Epoch 287
Validation binary_cross_entropy = 0.895782
Epoch 288
Validation binary_cross_entropy = 0.899061
Epoch 289
Loss = 5.4102e-04, PNorm = 94.7588, GNorm = 0.0800, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.894046
Epoch 290
Validation binary_cross_entropy = 0.892671
Epoch 291
Validation binary_cross_entropy = 0.892569
Epoch 292
Validation binary_cross_entropy = 0.896004
Epoch 293
Validation binary_cross_entropy = 0.910007
Epoch 294
Loss = 1.5820e-03, PNorm = 94.8001, GNorm = 3.7301, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.954253
Epoch 295
Validation binary_cross_entropy = 1.001379
Epoch 296
Validation binary_cross_entropy = 0.967263
Epoch 297
Validation binary_cross_entropy = 0.932262
Epoch 298
Validation binary_cross_entropy = 0.943842
Epoch 299
Loss = 4.3765e-06, PNorm = 94.9476, GNorm = 0.0140, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.945366
Model 0 best validation binary_cross_entropy = 0.298206 on epoch 1
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.264874
Ensemble test binary_cross_entropy = 0.264874
Fold 6
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_6',
 'save_smiles_splits': False,
 'seed': 6,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.401732
Best score so far is 0.401732 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.292252
Best score so far is 0.292252 at epoch 1 
Epoch 2
Validation binary_cross_entropy = 0.384402
Epoch 3
Validation binary_cross_entropy = 0.324324
Epoch 4
Loss = 8.8806e-03, PNorm = 87.7304, GNorm = 1.5035, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.598384
Epoch 5
Validation binary_cross_entropy = 0.471544
Epoch 6
Validation binary_cross_entropy = 0.437082
Epoch 7
Validation binary_cross_entropy = 0.276318
Best score so far is 0.276318 at epoch 7 
Epoch 8
Validation binary_cross_entropy = 0.613800
Epoch 9
Loss = 6.0623e-03, PNorm = 88.0477, GNorm = 2.5745, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.313230
Epoch 10
Validation binary_cross_entropy = 0.440353
Epoch 11
Validation binary_cross_entropy = 0.316768
Epoch 12
Validation binary_cross_entropy = 0.490189
Epoch 13
Validation binary_cross_entropy = 0.336650
Epoch 14
Loss = 5.4968e-03, PNorm = 88.3854, GNorm = 3.8293, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.348033
Epoch 15
Validation binary_cross_entropy = 0.410866
Epoch 16
Validation binary_cross_entropy = 0.401750
Epoch 17
Validation binary_cross_entropy = 0.382623
Epoch 18
Validation binary_cross_entropy = 0.449049
Epoch 19
Loss = 3.3949e-03, PNorm = 88.6358, GNorm = 1.7209, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.381595
Epoch 20
Validation binary_cross_entropy = 0.371230
Epoch 21
Validation binary_cross_entropy = 0.424877
Epoch 22
Validation binary_cross_entropy = 0.365317
Epoch 23
Validation binary_cross_entropy = 0.354277
Epoch 24
Loss = 3.9385e-03, PNorm = 88.8262, GNorm = 3.2562, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.432491
Epoch 25
Validation binary_cross_entropy = 0.454009
Epoch 26
Validation binary_cross_entropy = 0.414008
Epoch 27
Validation binary_cross_entropy = 0.391625
Epoch 28
Validation binary_cross_entropy = 0.535838
Epoch 29
Loss = 3.4361e-03, PNorm = 89.0081, GNorm = 1.3602, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.553584
Epoch 30
Validation binary_cross_entropy = 0.407916
Epoch 31
Validation binary_cross_entropy = 0.343382
Epoch 32
Validation binary_cross_entropy = 0.371644
Epoch 33
Validation binary_cross_entropy = 0.404284
Epoch 34
Loss = 5.3177e-03, PNorm = 89.1663, GNorm = 2.3276, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.379223
Epoch 35
Validation binary_cross_entropy = 0.365587
Epoch 36
Validation binary_cross_entropy = 0.426606
Epoch 37
Validation binary_cross_entropy = 0.458124
Epoch 38
Validation binary_cross_entropy = 0.407686
Epoch 39
Loss = 2.6636e-03, PNorm = 89.3551, GNorm = 1.3587, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.394573
Epoch 40
Validation binary_cross_entropy = 0.400241
Epoch 41
Validation binary_cross_entropy = 0.409541
Epoch 42
Validation binary_cross_entropy = 0.427429
Epoch 43
Validation binary_cross_entropy = 0.414374
Epoch 44
Loss = 1.4348e-03, PNorm = 89.5371, GNorm = 0.6654, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.407398
Epoch 45
Validation binary_cross_entropy = 0.419801
Epoch 46
Validation binary_cross_entropy = 0.429017
Epoch 47
Validation binary_cross_entropy = 0.441022
Epoch 48
Validation binary_cross_entropy = 0.451769
Epoch 49
Loss = 2.9224e-03, PNorm = 89.7388, GNorm = 1.7106, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.458959
Epoch 50
Validation binary_cross_entropy = 0.458862
Epoch 51
Validation binary_cross_entropy = 0.463955
Epoch 52
Validation binary_cross_entropy = 0.483672
Epoch 53
Validation binary_cross_entropy = 0.504057
Epoch 54
Loss = 1.5977e-03, PNorm = 89.9350, GNorm = 1.4581, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.603310
Epoch 55
Validation binary_cross_entropy = 0.483101
Epoch 56
Validation binary_cross_entropy = 0.570658
Epoch 57
Validation binary_cross_entropy = 0.475899
Epoch 58
Validation binary_cross_entropy = 0.611360
Epoch 59
Loss = 2.2630e-03, PNorm = 90.1520, GNorm = 2.0254, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.555421
Epoch 60
Validation binary_cross_entropy = 0.495037
Epoch 61
Validation binary_cross_entropy = 0.562321
Epoch 62
Validation binary_cross_entropy = 0.546322
Epoch 63
Validation binary_cross_entropy = 0.502883
Epoch 64
Loss = 2.4509e-03, PNorm = 90.3485, GNorm = 2.1091, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.490280
Epoch 65
Validation binary_cross_entropy = 0.460147
Epoch 66
Validation binary_cross_entropy = 0.456729
Epoch 67
Validation binary_cross_entropy = 0.461307
Epoch 68
Validation binary_cross_entropy = 0.478719
Epoch 69
Loss = 1.4243e-03, PNorm = 90.4971, GNorm = 1.4160, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.511970
Epoch 70
Validation binary_cross_entropy = 0.525761
Epoch 71
Validation binary_cross_entropy = 0.533261
Epoch 72
Validation binary_cross_entropy = 0.537235
Epoch 73
Validation binary_cross_entropy = 0.556519
Epoch 74
Loss = 1.0394e-03, PNorm = 90.6368, GNorm = 1.3699, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.555178
Epoch 75
Validation binary_cross_entropy = 0.559257
Epoch 76
Validation binary_cross_entropy = 0.564820
Epoch 77
Validation binary_cross_entropy = 0.581288
Epoch 78
Validation binary_cross_entropy = 0.612046
Epoch 79
Loss = 4.1566e-04, PNorm = 90.7763, GNorm = 0.2734, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.617546
Epoch 80
Validation binary_cross_entropy = 0.625944
Epoch 81
Validation binary_cross_entropy = 0.613788
Epoch 82
Validation binary_cross_entropy = 0.613197
Epoch 83
Validation binary_cross_entropy = 0.606090
Epoch 84
Loss = 8.5433e-04, PNorm = 90.9339, GNorm = 0.6249, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.578282
Epoch 85
Validation binary_cross_entropy = 0.558835
Epoch 86
Validation binary_cross_entropy = 0.554283
Epoch 87
Validation binary_cross_entropy = 0.559694
Epoch 88
Validation binary_cross_entropy = 0.591001
Epoch 89
Loss = 1.2354e-03, PNorm = 91.0599, GNorm = 1.9772, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.589787
Epoch 90
Validation binary_cross_entropy = 0.550570
Epoch 91
Validation binary_cross_entropy = 0.535147
Epoch 92
Validation binary_cross_entropy = 0.537315
Epoch 93
Validation binary_cross_entropy = 0.544573
Epoch 94
Loss = 3.3468e-04, PNorm = 91.1879, GNorm = 0.3161, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.558051
Epoch 95
Validation binary_cross_entropy = 0.578295
Epoch 96
Validation binary_cross_entropy = 0.589288
Epoch 97
Validation binary_cross_entropy = 0.563876
Epoch 98
Validation binary_cross_entropy = 0.561959
Epoch 99
Loss = 1.0849e-03, PNorm = 91.3387, GNorm = 1.7639, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.592559
Epoch 100
Validation binary_cross_entropy = 0.606939
Epoch 101
Validation binary_cross_entropy = 0.572498
Epoch 102
Validation binary_cross_entropy = 0.594380
Epoch 103
Validation binary_cross_entropy = 0.627584
Epoch 104
Loss = 1.0502e-04, PNorm = 91.4667, GNorm = 0.2435, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.646519
Epoch 105
Validation binary_cross_entropy = 0.664850
Epoch 106
Validation binary_cross_entropy = 0.667575
Epoch 107
Validation binary_cross_entropy = 0.659012
Epoch 108
Validation binary_cross_entropy = 0.663801
Epoch 109
Loss = 1.2014e-03, PNorm = 91.5756, GNorm = 0.9383, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.679426
Epoch 110
Validation binary_cross_entropy = 0.661839
Epoch 111
Validation binary_cross_entropy = 0.656628
Epoch 112
Validation binary_cross_entropy = 0.673993
Epoch 113
Validation binary_cross_entropy = 0.685208
Epoch 114
Loss = 7.2975e-05, PNorm = 91.6937, GNorm = 0.1224, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.689167
Epoch 115
Validation binary_cross_entropy = 0.716002
Epoch 116
Validation binary_cross_entropy = 0.758751
Epoch 117
Validation binary_cross_entropy = 0.764451
Epoch 118
Validation binary_cross_entropy = 0.713452
Epoch 119
Loss = 8.2927e-04, PNorm = 91.8222, GNorm = 0.0996, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.689895
Epoch 120
Validation binary_cross_entropy = 0.681722
Epoch 121
Validation binary_cross_entropy = 0.714068
Epoch 122
Validation binary_cross_entropy = 0.744395
Epoch 123
Validation binary_cross_entropy = 0.734961
Epoch 124
Loss = 4.3342e-04, PNorm = 91.9571, GNorm = 0.9960, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.693811
Epoch 125
Validation binary_cross_entropy = 0.699467
Epoch 126
Validation binary_cross_entropy = 0.802589
Epoch 127
Validation binary_cross_entropy = 0.652733
Epoch 128
Validation binary_cross_entropy = 0.609751
Epoch 129
Loss = 1.8399e-03, PNorm = 92.1730, GNorm = 2.4738, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.620994
Epoch 130
Validation binary_cross_entropy = 0.590150
Epoch 131
Validation binary_cross_entropy = 0.541250
Epoch 132
Validation binary_cross_entropy = 0.517387
Epoch 133
Validation binary_cross_entropy = 0.522367
Epoch 134
Loss = 1.4187e-03, PNorm = 92.4514, GNorm = 1.0781, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.533502
Epoch 135
Validation binary_cross_entropy = 0.555412
Epoch 136
Validation binary_cross_entropy = 0.566713
Epoch 137
Validation binary_cross_entropy = 0.562792
Epoch 138
Validation binary_cross_entropy = 0.572452
Epoch 139
Loss = 3.9319e-04, PNorm = 92.6465, GNorm = 0.3192, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.592534
Epoch 140
Validation binary_cross_entropy = 0.605188
Epoch 141
Validation binary_cross_entropy = 0.621998
Epoch 142
Validation binary_cross_entropy = 0.625328
Epoch 143
Validation binary_cross_entropy = 0.660381
Epoch 144
Loss = 7.9958e-04, PNorm = 92.7997, GNorm = 0.8668, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.656121
Epoch 145
Validation binary_cross_entropy = 0.626706
Epoch 146
Validation binary_cross_entropy = 0.625451
Epoch 147
Validation binary_cross_entropy = 0.614943
Epoch 148
Validation binary_cross_entropy = 0.587840
Epoch 149
Loss = 3.6466e-04, PNorm = 92.9607, GNorm = 1.2815, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.608546
Epoch 150
Validation binary_cross_entropy = 0.554255
Epoch 151
Validation binary_cross_entropy = 0.615105
Epoch 152
Validation binary_cross_entropy = 0.549404
Epoch 153
Validation binary_cross_entropy = 0.544729
Epoch 154
Loss = 1.0502e-03, PNorm = 93.1721, GNorm = 0.3556, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.568392
Epoch 155
Validation binary_cross_entropy = 0.554464
Epoch 156
Validation binary_cross_entropy = 0.535835
Epoch 157
Validation binary_cross_entropy = 0.554532
Epoch 158
Validation binary_cross_entropy = 0.567143
Epoch 159
Loss = 6.4745e-04, PNorm = 93.3650, GNorm = 1.6537, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.552214
Epoch 160
Validation binary_cross_entropy = 0.551699
Epoch 161
Validation binary_cross_entropy = 0.577205
Epoch 162
Validation binary_cross_entropy = 0.650237
Epoch 163
Validation binary_cross_entropy = 0.683045
Epoch 164
Loss = 5.7576e-04, PNorm = 93.5142, GNorm = 0.5835, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.646070
Epoch 165
Validation binary_cross_entropy = 0.615275
Epoch 166
Validation binary_cross_entropy = 0.608587
Epoch 167
Validation binary_cross_entropy = 0.603562
Epoch 168
Validation binary_cross_entropy = 0.604971
Epoch 169
Loss = 7.5117e-04, PNorm = 93.6315, GNorm = 0.5533, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.603744
Epoch 170
Validation binary_cross_entropy = 0.632262
Epoch 171
Validation binary_cross_entropy = 0.674847
Epoch 172
Validation binary_cross_entropy = 0.688724
Epoch 173
Validation binary_cross_entropy = 0.661823
Epoch 174
Loss = 5.9281e-04, PNorm = 93.7151, GNorm = 0.3047, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.620129
Epoch 175
Validation binary_cross_entropy = 0.617274
Epoch 176
Validation binary_cross_entropy = 0.626742
Epoch 177
Validation binary_cross_entropy = 0.625587
Epoch 178
Validation binary_cross_entropy = 0.611783
Epoch 179
Loss = 9.0715e-04, PNorm = 93.7902, GNorm = 1.8825, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.623372
Epoch 180
Validation binary_cross_entropy = 0.676070
Epoch 181
Validation binary_cross_entropy = 0.686749
Epoch 182
Validation binary_cross_entropy = 0.653656
Epoch 183
Validation binary_cross_entropy = 0.637331
Epoch 184
Loss = 3.7109e-05, PNorm = 93.8973, GNorm = 0.0530, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.649872
Epoch 185
Validation binary_cross_entropy = 0.654897
Epoch 186
Validation binary_cross_entropy = 0.618898
Epoch 187
Validation binary_cross_entropy = 0.654742
Epoch 188
Validation binary_cross_entropy = 0.692264
Epoch 189
Loss = 6.3059e-04, PNorm = 93.9731, GNorm = 2.0921, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.687944
Epoch 190
Validation binary_cross_entropy = 0.637538
Epoch 191
Validation binary_cross_entropy = 0.596069
Epoch 192
Validation binary_cross_entropy = 0.576690
Epoch 193
Validation binary_cross_entropy = 0.567829
Epoch 194
Loss = 5.9956e-04, PNorm = 94.0786, GNorm = 0.9898, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.570008
Epoch 195
Validation binary_cross_entropy = 0.586246
Epoch 196
Validation binary_cross_entropy = 0.617393
Epoch 197
Validation binary_cross_entropy = 0.659653
Epoch 198
Validation binary_cross_entropy = 0.633794
Epoch 199
Loss = 2.4462e-04, PNorm = 94.1526, GNorm = 0.3135, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.594538
Epoch 200
Validation binary_cross_entropy = 0.577648
Epoch 201
Validation binary_cross_entropy = 0.571103
Epoch 202
Validation binary_cross_entropy = 0.569363
Epoch 203
Validation binary_cross_entropy = 0.568119
Epoch 204
Loss = 2.8032e-04, PNorm = 94.2179, GNorm = 0.3756, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.579237
Epoch 205
Validation binary_cross_entropy = 0.598580
Epoch 206
Validation binary_cross_entropy = 0.621737
Epoch 207
Validation binary_cross_entropy = 0.635520
Epoch 208
Validation binary_cross_entropy = 0.645585
Epoch 209
Loss = 4.0628e-05, PNorm = 94.2607, GNorm = 0.0895, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.651647
Epoch 210
Validation binary_cross_entropy = 0.656238
Epoch 211
Validation binary_cross_entropy = 0.637407
Epoch 212
Validation binary_cross_entropy = 0.615715
Epoch 213
Validation binary_cross_entropy = 0.601054
Epoch 214
Loss = 1.8744e-04, PNorm = 94.3001, GNorm = 0.0778, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.591226
Epoch 215
Validation binary_cross_entropy = 0.602512
Epoch 216
Validation binary_cross_entropy = 0.647587
Epoch 217
Validation binary_cross_entropy = 0.693429
Epoch 218
Validation binary_cross_entropy = 0.675372
Epoch 219
Loss = 1.0951e-04, PNorm = 94.3656, GNorm = 0.0691, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.654988
Epoch 220
Validation binary_cross_entropy = 0.644630
Epoch 221
Validation binary_cross_entropy = 0.646711
Epoch 222
Validation binary_cross_entropy = 0.654445
Epoch 223
Validation binary_cross_entropy = 0.657547
Epoch 224
Loss = 1.4772e-04, PNorm = 94.4214, GNorm = 0.2128, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.664040
Epoch 225
Validation binary_cross_entropy = 0.676057
Epoch 226
Validation binary_cross_entropy = 0.690288
Epoch 227
Validation binary_cross_entropy = 0.703910
Epoch 228
Validation binary_cross_entropy = 0.731361
Epoch 229
Loss = 3.5138e-05, PNorm = 94.4596, GNorm = 0.1186, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.755948
Epoch 230
Validation binary_cross_entropy = 0.766092
Epoch 231
Validation binary_cross_entropy = 0.760086
Epoch 232
Validation binary_cross_entropy = 0.749052
Epoch 233
Validation binary_cross_entropy = 0.740232
Epoch 234
Loss = 1.6248e-04, PNorm = 94.4929, GNorm = 0.1201, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.732441
Epoch 235
Validation binary_cross_entropy = 0.744425
Epoch 236
Validation binary_cross_entropy = 0.774998
Epoch 237
Validation binary_cross_entropy = 0.800762
Epoch 238
Validation binary_cross_entropy = 0.806412
Epoch 239
Loss = 1.4269e-05, PNorm = 94.5272, GNorm = 0.0258, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.798982
Epoch 240
Validation binary_cross_entropy = 0.794267
Epoch 241
Validation binary_cross_entropy = 0.805646
Epoch 242
Validation binary_cross_entropy = 0.819606
Epoch 243
Validation binary_cross_entropy = 0.830298
Epoch 244
Loss = 2.2910e-05, PNorm = 94.5769, GNorm = 0.0742, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.841393
Epoch 245
Validation binary_cross_entropy = 0.847979
Epoch 246
Validation binary_cross_entropy = 0.845259
Epoch 247
Validation binary_cross_entropy = 0.840522
Epoch 248
Validation binary_cross_entropy = 0.836569
Epoch 249
Loss = 7.9759e-04, PNorm = 94.6307, GNorm = 0.0148, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.845801
Epoch 250
Validation binary_cross_entropy = 0.857036
Epoch 251
Validation binary_cross_entropy = 0.868733
Epoch 252
Validation binary_cross_entropy = 0.855719
Epoch 253
Validation binary_cross_entropy = 0.798070
Epoch 254
Loss = 7.3829e-05, PNorm = 94.7008, GNorm = 0.1012, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.800240
Epoch 255
Validation binary_cross_entropy = 0.821088
Epoch 256
Validation binary_cross_entropy = 0.828870
Epoch 257
Validation binary_cross_entropy = 0.819825
Epoch 258
Validation binary_cross_entropy = 0.858277
Epoch 259
Loss = 1.9939e-05, PNorm = 94.8292, GNorm = 0.0628, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.909674
Epoch 260
Validation binary_cross_entropy = 0.907965
Epoch 261
Validation binary_cross_entropy = 0.893121
Epoch 262
Validation binary_cross_entropy = 0.838813
Epoch 263
Validation binary_cross_entropy = 0.800001
Epoch 264
Loss = 6.1908e-05, PNorm = 95.0110, GNorm = 0.0344, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.769002
Epoch 265
Validation binary_cross_entropy = 0.756954
Epoch 266
Validation binary_cross_entropy = 0.753893
Epoch 267
Validation binary_cross_entropy = 0.754942
Epoch 268
Validation binary_cross_entropy = 0.756595
Epoch 269
Loss = 9.5133e-05, PNorm = 95.1335, GNorm = 0.0794, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.764421
Epoch 270
Validation binary_cross_entropy = 0.766310
Epoch 271
Validation binary_cross_entropy = 0.777795
Epoch 272
Validation binary_cross_entropy = 0.797788
Epoch 273
Validation binary_cross_entropy = 0.851819
Epoch 274
Loss = 2.3585e-04, PNorm = 95.2049, GNorm = 0.8208, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.898102
Epoch 275
Validation binary_cross_entropy = 0.876126
Epoch 276
Validation binary_cross_entropy = 0.832714
Epoch 277
Validation binary_cross_entropy = 0.794610
Epoch 278
Validation binary_cross_entropy = 0.776095
Epoch 279
Loss = 4.9205e-05, PNorm = 95.2996, GNorm = 0.0686, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.774665
Epoch 280
Validation binary_cross_entropy = 0.762570
Epoch 281
Validation binary_cross_entropy = 0.708137
Epoch 282
Validation binary_cross_entropy = 0.725416
Epoch 283
Validation binary_cross_entropy = 0.744658
Epoch 284
Loss = 8.8393e-05, PNorm = 95.4152, GNorm = 0.3265, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.765124
Epoch 285
Validation binary_cross_entropy = 0.777575
Epoch 286
Validation binary_cross_entropy = 0.781238
Epoch 287
Validation binary_cross_entropy = 0.788415
Epoch 288
Validation binary_cross_entropy = 0.789832
Epoch 289
Loss = 3.4603e-04, PNorm = 95.5120, GNorm = 0.0674, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.784070
Epoch 290
Validation binary_cross_entropy = 0.782232
Epoch 291
Validation binary_cross_entropy = 0.793843
Epoch 292
Validation binary_cross_entropy = 0.800700
Epoch 293
Validation binary_cross_entropy = 0.790524
Epoch 294
Loss = 1.8613e-04, PNorm = 95.5766, GNorm = 0.2586, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.770976
Epoch 295
Validation binary_cross_entropy = 0.761129
Epoch 296
Validation binary_cross_entropy = 0.766173
Epoch 297
Validation binary_cross_entropy = 0.772125
Epoch 298
Validation binary_cross_entropy = 0.770871
Epoch 299
Loss = 2.9761e-05, PNorm = 95.6171, GNorm = 0.0745, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.770680
Model 0 best validation binary_cross_entropy = 0.276318 on epoch 7
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.182453
Ensemble test binary_cross_entropy = 0.182453
Fold 7
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_7',
 'save_smiles_splits': False,
 'seed': 7,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.281440
Best score so far is 0.281440 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.398285
Epoch 2
Validation binary_cross_entropy = 0.399566
Epoch 3
Validation binary_cross_entropy = 0.353874
Epoch 4
Loss = 1.1340e-02, PNorm = 87.7223, GNorm = 1.9095, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.269997
Best score so far is 0.269997 at epoch 4 
Epoch 5
Validation binary_cross_entropy = 0.748447
Epoch 6
Validation binary_cross_entropy = 0.285181
Epoch 7
Validation binary_cross_entropy = 0.413213
Epoch 8
Validation binary_cross_entropy = 0.354484
Epoch 9
Loss = 6.0532e-03, PNorm = 88.0199, GNorm = 1.0417, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.512157
Epoch 10
Validation binary_cross_entropy = 0.307343
Epoch 11
Validation binary_cross_entropy = 0.434727
Epoch 12
Validation binary_cross_entropy = 0.400864
Epoch 13
Validation binary_cross_entropy = 0.338503
Epoch 14
Loss = 3.3619e-03, PNorm = 88.3386, GNorm = 1.3985, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.390931
Epoch 15
Validation binary_cross_entropy = 0.333912
Epoch 16
Validation binary_cross_entropy = 0.376833
Epoch 17
Validation binary_cross_entropy = 0.344026
Epoch 18
Validation binary_cross_entropy = 0.340218
Epoch 19
Loss = 2.6019e-03, PNorm = 88.5755, GNorm = 0.9132, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.452506
Epoch 20
Validation binary_cross_entropy = 0.427170
Epoch 21
Validation binary_cross_entropy = 0.451409
Epoch 22
Validation binary_cross_entropy = 0.491913
Epoch 23
Validation binary_cross_entropy = 0.511774
Epoch 24
Loss = 2.0980e-03, PNorm = 88.7698, GNorm = 1.8856, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.486291
Epoch 25
Validation binary_cross_entropy = 0.478217
Epoch 26
Validation binary_cross_entropy = 0.501363
Epoch 27
Validation binary_cross_entropy = 0.477993
Epoch 28
Validation binary_cross_entropy = 0.418554
Epoch 29
Loss = 2.4294e-03, PNorm = 88.9551, GNorm = 1.4557, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.397007
Epoch 30
Validation binary_cross_entropy = 0.394677
Epoch 31
Validation binary_cross_entropy = 0.441909
Epoch 32
Validation binary_cross_entropy = 0.456265
Epoch 33
Validation binary_cross_entropy = 0.466761
Epoch 34
Loss = 4.0479e-03, PNorm = 89.1391, GNorm = 1.5257, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.426289
Epoch 35
Validation binary_cross_entropy = 0.407901
Epoch 36
Validation binary_cross_entropy = 0.393736
Epoch 37
Validation binary_cross_entropy = 0.420789
Epoch 38
Validation binary_cross_entropy = 0.453367
Epoch 39
Loss = 1.2173e-03, PNorm = 89.3478, GNorm = 1.1411, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.449055
Epoch 40
Validation binary_cross_entropy = 0.479763
Epoch 41
Validation binary_cross_entropy = 0.458758
Epoch 42
Validation binary_cross_entropy = 0.458394
Epoch 43
Validation binary_cross_entropy = 0.464389
Epoch 44
Loss = 2.2371e-03, PNorm = 89.5111, GNorm = 2.1949, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.509229
Epoch 45
Validation binary_cross_entropy = 0.466799
Epoch 46
Validation binary_cross_entropy = 0.475573
Epoch 47
Validation binary_cross_entropy = 0.502192
Epoch 48
Validation binary_cross_entropy = 0.507571
Epoch 49
Loss = 1.2989e-03, PNorm = 89.6643, GNorm = 1.0977, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.477808
Epoch 50
Validation binary_cross_entropy = 0.513016
Epoch 51
Validation binary_cross_entropy = 0.479780
Epoch 52
Validation binary_cross_entropy = 0.427629
Epoch 53
Validation binary_cross_entropy = 0.452515
Epoch 54
Loss = 1.3617e-03, PNorm = 89.8481, GNorm = 0.5476, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.465886
Epoch 55
Validation binary_cross_entropy = 0.414635
Epoch 56
Validation binary_cross_entropy = 0.398233
Epoch 57
Validation binary_cross_entropy = 0.414060
Epoch 58
Validation binary_cross_entropy = 0.435411
Epoch 59
Loss = 2.2355e-03, PNorm = 90.0353, GNorm = 0.7273, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.382121
Epoch 60
Validation binary_cross_entropy = 0.379754
Epoch 61
Validation binary_cross_entropy = 0.392790
Epoch 62
Validation binary_cross_entropy = 0.398866
Epoch 63
Validation binary_cross_entropy = 0.452468
Epoch 64
Loss = 7.8015e-04, PNorm = 90.2133, GNorm = 1.7188, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.509234
Epoch 65
Validation binary_cross_entropy = 0.654161
Epoch 66
Validation binary_cross_entropy = 0.552724
Epoch 67
Validation binary_cross_entropy = 0.548624
Epoch 68
Validation binary_cross_entropy = 0.490770
Epoch 69
Loss = 2.6649e-03, PNorm = 90.4249, GNorm = 1.8082, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.503567
Epoch 70
Validation binary_cross_entropy = 0.497075
Epoch 71
Validation binary_cross_entropy = 0.489114
Epoch 72
Validation binary_cross_entropy = 0.552416
Epoch 73
Validation binary_cross_entropy = 0.527327
Epoch 74
Loss = 2.5374e-03, PNorm = 90.6733, GNorm = 2.9389, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.514228
Epoch 75
Validation binary_cross_entropy = 0.467117
Epoch 76
Validation binary_cross_entropy = 0.461526
Epoch 77
Validation binary_cross_entropy = 0.488844
Epoch 78
Validation binary_cross_entropy = 0.442000
Epoch 79
Loss = 8.6185e-04, PNorm = 90.8854, GNorm = 1.1820, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.408055
Epoch 80
Validation binary_cross_entropy = 0.420239
Epoch 81
Validation binary_cross_entropy = 0.454510
Epoch 82
Validation binary_cross_entropy = 0.480843
Epoch 83
Validation binary_cross_entropy = 0.467424
Epoch 84
Loss = 1.7771e-03, PNorm = 91.0913, GNorm = 0.9429, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.446810
Epoch 85
Validation binary_cross_entropy = 0.456549
Epoch 86
Validation binary_cross_entropy = 0.497316
Epoch 87
Validation binary_cross_entropy = 0.539661
Epoch 88
Validation binary_cross_entropy = 0.550982
Epoch 89
Loss = 4.2759e-04, PNorm = 91.2763, GNorm = 0.5860, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.557641
Epoch 90
Validation binary_cross_entropy = 0.585212
Epoch 91
Validation binary_cross_entropy = 0.596965
Epoch 92
Validation binary_cross_entropy = 0.611580
Epoch 93
Validation binary_cross_entropy = 0.660463
Epoch 94
Loss = 9.4951e-04, PNorm = 91.4012, GNorm = 2.2524, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.691710
Epoch 95
Validation binary_cross_entropy = 0.684972
Epoch 96
Validation binary_cross_entropy = 0.676446
Epoch 97
Validation binary_cross_entropy = 0.692724
Epoch 98
Validation binary_cross_entropy = 0.621231
Epoch 99
Loss = 1.1923e-03, PNorm = 91.5533, GNorm = 1.0530, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.587303
Epoch 100
Validation binary_cross_entropy = 0.594491
Epoch 101
Validation binary_cross_entropy = 0.611855
Epoch 102
Validation binary_cross_entropy = 0.632366
Epoch 103
Validation binary_cross_entropy = 0.665592
Epoch 104
Loss = 4.0056e-04, PNorm = 91.7525, GNorm = 0.6249, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.655454
Epoch 105
Validation binary_cross_entropy = 0.590876
Epoch 106
Validation binary_cross_entropy = 0.574388
Epoch 107
Validation binary_cross_entropy = 0.568859
Epoch 108
Validation binary_cross_entropy = 0.561079
Epoch 109
Loss = 9.0692e-04, PNorm = 91.9339, GNorm = 1.7898, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.576376
Epoch 110
Validation binary_cross_entropy = 0.583380
Epoch 111
Validation binary_cross_entropy = 0.595542
Epoch 112
Validation binary_cross_entropy = 0.613912
Epoch 113
Validation binary_cross_entropy = 0.628421
Epoch 114
Loss = 1.0570e-03, PNorm = 92.0839, GNorm = 0.6799, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.582175
Epoch 115
Validation binary_cross_entropy = 0.584332
Epoch 116
Validation binary_cross_entropy = 0.613790
Epoch 117
Validation binary_cross_entropy = 0.712566
Epoch 118
Validation binary_cross_entropy = 0.709179
Epoch 119
Loss = 6.9760e-04, PNorm = 92.2385, GNorm = 1.1217, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.643250
Epoch 120
Validation binary_cross_entropy = 0.609041
Epoch 121
Validation binary_cross_entropy = 0.615196
Epoch 122
Validation binary_cross_entropy = 0.625705
Epoch 123
Validation binary_cross_entropy = 0.643199
Epoch 124
Loss = 2.4483e-04, PNorm = 92.3683, GNorm = 0.2975, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.660152
Epoch 125
Validation binary_cross_entropy = 0.627224
Epoch 126
Validation binary_cross_entropy = 0.601739
Epoch 127
Validation binary_cross_entropy = 0.589619
Epoch 128
Validation binary_cross_entropy = 0.596033
Epoch 129
Loss = 2.3090e-04, PNorm = 92.4754, GNorm = 0.5506, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.611598
Epoch 130
Validation binary_cross_entropy = 0.616487
Epoch 131
Validation binary_cross_entropy = 0.623209
Epoch 132
Validation binary_cross_entropy = 0.634397
Epoch 133
Validation binary_cross_entropy = 0.650882
Epoch 134
Loss = 3.0627e-04, PNorm = 92.5835, GNorm = 0.5962, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.666174
Epoch 135
Validation binary_cross_entropy = 0.679221
Epoch 136
Validation binary_cross_entropy = 0.683234
Epoch 137
Validation binary_cross_entropy = 0.690681
Epoch 138
Validation binary_cross_entropy = 0.696713
Epoch 139
Loss = 9.4719e-04, PNorm = 92.6690, GNorm = 0.0835, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.676623
Epoch 140
Validation binary_cross_entropy = 0.675287
Epoch 141
Validation binary_cross_entropy = 0.683077
Epoch 142
Validation binary_cross_entropy = 0.715780
Epoch 143
Validation binary_cross_entropy = 0.743496
Epoch 144
Loss = 7.6447e-05, PNorm = 92.7311, GNorm = 0.1754, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.756673
Epoch 145
Validation binary_cross_entropy = 0.757419
Epoch 146
Validation binary_cross_entropy = 0.745262
Epoch 147
Validation binary_cross_entropy = 0.733728
Epoch 148
Validation binary_cross_entropy = 0.727835
Epoch 149
Loss = 8.7834e-05, PNorm = 92.7927, GNorm = 0.3455, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.725566
Epoch 150
Validation binary_cross_entropy = 0.720430
Epoch 151
Validation binary_cross_entropy = 0.721941
Epoch 152
Validation binary_cross_entropy = 0.728648
Epoch 153
Validation binary_cross_entropy = 0.736152
Epoch 154
Loss = 6.2773e-05, PNorm = 92.8448, GNorm = 0.4291, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.743447
Epoch 155
Validation binary_cross_entropy = 0.793706
Epoch 156
Validation binary_cross_entropy = 0.879821
Epoch 157
Validation binary_cross_entropy = 0.843936
Epoch 158
Validation binary_cross_entropy = 0.800021
Epoch 159
Loss = 1.2594e-03, PNorm = 92.8970, GNorm = 2.3417, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.793816
Epoch 160
Validation binary_cross_entropy = 0.839857
Epoch 161
Validation binary_cross_entropy = 0.865278
Epoch 162
Validation binary_cross_entropy = 0.842059
Epoch 163
Validation binary_cross_entropy = 0.836058
Epoch 164
Loss = 2.7434e-04, PNorm = 93.0268, GNorm = 0.8170, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.876181
Epoch 165
Validation binary_cross_entropy = 0.901671
Epoch 166
Validation binary_cross_entropy = 0.958026
Epoch 167
Validation binary_cross_entropy = 1.005042
Epoch 168
Validation binary_cross_entropy = 1.024238
Epoch 169
Loss = 2.8307e-04, PNorm = 93.1330, GNorm = 0.3502, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.046190
Epoch 170
Validation binary_cross_entropy = 0.884937
Epoch 171
Validation binary_cross_entropy = 0.825388
Epoch 172
Validation binary_cross_entropy = 0.830387
Epoch 173
Validation binary_cross_entropy = 0.875454
Epoch 174
Loss = 2.2110e-04, PNorm = 93.2675, GNorm = 0.0673, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.842997
Epoch 175
Validation binary_cross_entropy = 0.885643
Epoch 176
Validation binary_cross_entropy = 0.904213
Epoch 177
Validation binary_cross_entropy = 0.766058
Epoch 178
Validation binary_cross_entropy = 0.740705
Epoch 179
Loss = 1.2685e-04, PNorm = 93.4382, GNorm = 0.5089, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.802951
Epoch 180
Validation binary_cross_entropy = 0.697298
Epoch 181
Validation binary_cross_entropy = 0.674897
Epoch 182
Validation binary_cross_entropy = 0.651143
Epoch 183
Validation binary_cross_entropy = 0.655873
Epoch 184
Loss = 2.0503e-03, PNorm = 93.6713, GNorm = 3.5142, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.572663
Epoch 185
Validation binary_cross_entropy = 0.512040
Epoch 186
Validation binary_cross_entropy = 0.548303
Epoch 187
Validation binary_cross_entropy = 0.633471
Epoch 188
Validation binary_cross_entropy = 0.650871
Epoch 189
Loss = 1.1003e-04, PNorm = 94.0107, GNorm = 0.0367, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.693476
Epoch 190
Validation binary_cross_entropy = 0.798252
Epoch 191
Validation binary_cross_entropy = 0.937102
Epoch 192
Validation binary_cross_entropy = 0.994798
Epoch 193
Validation binary_cross_entropy = 0.963992
Epoch 194
Loss = 9.9562e-05, PNorm = 94.2455, GNorm = 0.2483, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.903108
Epoch 195
Validation binary_cross_entropy = 0.980975
Epoch 196
Validation binary_cross_entropy = 0.840146
Epoch 197
Validation binary_cross_entropy = 0.698210
Epoch 198
Validation binary_cross_entropy = 0.667146
Epoch 199
Loss = 1.0826e-03, PNorm = 94.4084, GNorm = 2.8009, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.663775
Epoch 200
Validation binary_cross_entropy = 0.673219
Epoch 201
Validation binary_cross_entropy = 0.711278
Epoch 202
Validation binary_cross_entropy = 0.733075
Epoch 203
Validation binary_cross_entropy = 0.774570
Epoch 204
Loss = 1.0531e-04, PNorm = 94.5517, GNorm = 0.5336, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.808739
Epoch 205
Validation binary_cross_entropy = 0.795797
Epoch 206
Validation binary_cross_entropy = 0.790665
Epoch 207
Validation binary_cross_entropy = 0.795162
Epoch 208
Validation binary_cross_entropy = 0.791696
Epoch 209
Loss = 3.6878e-05, PNorm = 94.6809, GNorm = 0.0173, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.786477
Epoch 210
Validation binary_cross_entropy = 0.777700
Epoch 211
Validation binary_cross_entropy = 0.772455
Epoch 212
Validation binary_cross_entropy = 0.773290
Epoch 213
Validation binary_cross_entropy = 0.762453
Epoch 214
Loss = 7.6669e-05, PNorm = 94.7443, GNorm = 0.0963, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.756267
Epoch 215
Validation binary_cross_entropy = 0.770680
Epoch 216
Validation binary_cross_entropy = 0.771102
Epoch 217
Validation binary_cross_entropy = 0.781175
Epoch 218
Validation binary_cross_entropy = 0.769111
Epoch 219
Loss = 1.4713e-04, PNorm = 94.8224, GNorm = 0.5644, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.731579
Epoch 220
Validation binary_cross_entropy = 0.693657
Epoch 221
Validation binary_cross_entropy = 0.677861
Epoch 222
Validation binary_cross_entropy = 0.696056
Epoch 223
Validation binary_cross_entropy = 0.709902
Epoch 224
Loss = 5.7887e-04, PNorm = 94.9533, GNorm = 0.9672, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.884342
Epoch 225
Validation binary_cross_entropy = 1.015144
Epoch 226
Validation binary_cross_entropy = 0.801281
Epoch 227
Validation binary_cross_entropy = 0.729773
Epoch 228
Validation binary_cross_entropy = 0.737674
Epoch 229
Loss = 5.2262e-04, PNorm = 95.1189, GNorm = 1.3370, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.739517
Epoch 230
Validation binary_cross_entropy = 0.722025
Epoch 231
Validation binary_cross_entropy = 0.725633
Epoch 232
Validation binary_cross_entropy = 0.737151
Epoch 233
Validation binary_cross_entropy = 0.693275
Epoch 234
Loss = 4.9836e-05, PNorm = 95.2696, GNorm = 0.0512, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.651305
Epoch 235
Validation binary_cross_entropy = 0.630662
Epoch 236
Validation binary_cross_entropy = 0.637977
Epoch 237
Validation binary_cross_entropy = 0.671255
Epoch 238
Validation binary_cross_entropy = 0.700332
Epoch 239
Loss = 5.7263e-05, PNorm = 95.3702, GNorm = 0.1490, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.721028
Epoch 240
Validation binary_cross_entropy = 0.723391
Epoch 241
Validation binary_cross_entropy = 0.660726
Epoch 242
Validation binary_cross_entropy = 0.651865
Epoch 243
Validation binary_cross_entropy = 0.664760
Epoch 244
Loss = 3.5509e-05, PNorm = 95.4379, GNorm = 0.0058, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.672999
Epoch 245
Validation binary_cross_entropy = 0.694956
Epoch 246
Validation binary_cross_entropy = 0.693903
Epoch 247
Validation binary_cross_entropy = 0.665914
Epoch 248
Validation binary_cross_entropy = 0.652642
Epoch 249
Loss = 1.1933e-04, PNorm = 95.5147, GNorm = 0.5243, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.653168
Epoch 250
Validation binary_cross_entropy = 0.631012
Epoch 251
Validation binary_cross_entropy = 0.620217
Epoch 252
Validation binary_cross_entropy = 0.627700
Epoch 253
Validation binary_cross_entropy = 0.660475
Epoch 254
Loss = 3.5620e-05, PNorm = 95.5776, GNorm = 0.0163, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.695432
Epoch 255
Validation binary_cross_entropy = 0.727307
Epoch 256
Validation binary_cross_entropy = 0.751626
Epoch 257
Validation binary_cross_entropy = 0.740199
Epoch 258
Validation binary_cross_entropy = 0.732080
Epoch 259
Loss = 1.2781e-04, PNorm = 95.6243, GNorm = 0.0235, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.748790
Epoch 260
Validation binary_cross_entropy = 0.777743
Epoch 261
Validation binary_cross_entropy = 0.811827
Epoch 262
Validation binary_cross_entropy = 0.847191
Epoch 263
Validation binary_cross_entropy = 0.871721
Epoch 264
Loss = 2.2669e-05, PNorm = 95.6807, GNorm = 0.0980, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.937851
Epoch 265
Validation binary_cross_entropy = 0.971936
Epoch 266
Validation binary_cross_entropy = 0.869693
Epoch 267
Validation binary_cross_entropy = 0.774425
Epoch 268
Validation binary_cross_entropy = 0.739307
Epoch 269
Loss = 1.2216e-04, PNorm = 95.7777, GNorm = 0.0756, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.735848
Epoch 270
Validation binary_cross_entropy = 0.735890
Epoch 271
Validation binary_cross_entropy = 0.725933
Epoch 272
Validation binary_cross_entropy = 0.720014
Epoch 273
Validation binary_cross_entropy = 0.719377
Epoch 274
Loss = 1.5492e-04, PNorm = 95.8717, GNorm = 0.7469, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.711806
Epoch 275
Validation binary_cross_entropy = 0.707159
Epoch 276
Validation binary_cross_entropy = 0.710894
Epoch 277
Validation binary_cross_entropy = 0.719872
Epoch 278
Validation binary_cross_entropy = 0.727129
Epoch 279
Loss = 5.9506e-06, PNorm = 95.9303, GNorm = 0.0143, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.737770
Epoch 280
Validation binary_cross_entropy = 0.747205
Epoch 281
Validation binary_cross_entropy = 0.755801
Epoch 282
Validation binary_cross_entropy = 0.766675
Epoch 283
Validation binary_cross_entropy = 0.775342
Epoch 284
Loss = 4.6334e-05, PNorm = 95.9646, GNorm = 0.3075, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.784941
Epoch 285
Validation binary_cross_entropy = 0.795114
Epoch 286
Validation binary_cross_entropy = 0.804729
Epoch 287
Validation binary_cross_entropy = 0.812835
Epoch 288
Validation binary_cross_entropy = 0.819176
Epoch 289
Loss = 1.8862e-05, PNorm = 95.9841, GNorm = 0.0375, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.824268
Epoch 290
Validation binary_cross_entropy = 0.828143
Epoch 291
Validation binary_cross_entropy = 0.832237
Epoch 292
Validation binary_cross_entropy = 0.833479
Epoch 293
Validation binary_cross_entropy = 0.851727
Epoch 294
Loss = 2.4171e-04, PNorm = 96.0079, GNorm = 1.2732, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.862068
Epoch 295
Validation binary_cross_entropy = 0.802919
Epoch 296
Validation binary_cross_entropy = 0.791424
Epoch 297
Validation binary_cross_entropy = 0.809504
Epoch 298
Validation binary_cross_entropy = 0.837207
Epoch 299
Loss = 1.4269e-04, PNorm = 96.0491, GNorm = 1.5620, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.868811
Model 0 best validation binary_cross_entropy = 0.269997 on epoch 4
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.164410
Ensemble test binary_cross_entropy = 0.164410
Fold 8
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_8',
 'save_smiles_splits': False,
 'seed': 8,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.278535
Best score so far is 0.278535 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.485844
Epoch 2
Validation binary_cross_entropy = 0.372886
Epoch 3
Validation binary_cross_entropy = 0.414086
Epoch 4
Loss = 7.1909e-03, PNorm = 87.7287, GNorm = 2.0511, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.358871
Epoch 5
Validation binary_cross_entropy = 0.756226
Epoch 6
Validation binary_cross_entropy = 0.315784
Epoch 7
Validation binary_cross_entropy = 0.544517
Epoch 8
Validation binary_cross_entropy = 0.491092
Epoch 9
Loss = 7.5585e-03, PNorm = 88.0314, GNorm = 2.8224, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.331782
Epoch 10
Validation binary_cross_entropy = 0.486712
Epoch 11
Validation binary_cross_entropy = 0.478615
Epoch 12
Validation binary_cross_entropy = 0.441781
Epoch 13
Validation binary_cross_entropy = 0.519380
Epoch 14
Loss = 6.2110e-03, PNorm = 88.3619, GNorm = 0.8121, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.441419
Epoch 15
Validation binary_cross_entropy = 0.404648
Epoch 16
Validation binary_cross_entropy = 0.342504
Epoch 17
Validation binary_cross_entropy = 0.386491
Epoch 18
Validation binary_cross_entropy = 0.445624
Epoch 19
Loss = 3.4733e-03, PNorm = 88.6120, GNorm = 0.7584, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.370285
Epoch 20
Validation binary_cross_entropy = 0.355272
Epoch 21
Validation binary_cross_entropy = 0.387171
Epoch 22
Validation binary_cross_entropy = 0.401913
Epoch 23
Validation binary_cross_entropy = 0.345386
Epoch 24
Loss = 3.2613e-03, PNorm = 88.8037, GNorm = 2.9783, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.400589
Epoch 25
Validation binary_cross_entropy = 0.591884
Epoch 26
Validation binary_cross_entropy = 0.448757
Epoch 27
Validation binary_cross_entropy = 0.416118
Epoch 28
Validation binary_cross_entropy = 0.439615
Epoch 29
Loss = 4.1128e-03, PNorm = 88.9693, GNorm = 3.5081, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.444142
Epoch 30
Validation binary_cross_entropy = 0.401665
Epoch 31
Validation binary_cross_entropy = 0.442470
Epoch 32
Validation binary_cross_entropy = 0.460122
Epoch 33
Validation binary_cross_entropy = 0.441359
Epoch 34
Loss = 3.2845e-03, PNorm = 89.1327, GNorm = 0.9401, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.423187
Epoch 35
Validation binary_cross_entropy = 0.417774
Epoch 36
Validation binary_cross_entropy = 0.409077
Epoch 37
Validation binary_cross_entropy = 0.425549
Epoch 38
Validation binary_cross_entropy = 0.464760
Epoch 39
Loss = 4.3071e-03, PNorm = 89.2902, GNorm = 3.8826, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.420182
Epoch 40
Validation binary_cross_entropy = 0.489510
Epoch 41
Validation binary_cross_entropy = 0.412518
Epoch 42
Validation binary_cross_entropy = 0.418380
Epoch 43
Validation binary_cross_entropy = 0.439552
Epoch 44
Loss = 1.9117e-03, PNorm = 89.4789, GNorm = 1.0242, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.440315
Epoch 45
Validation binary_cross_entropy = 0.433523
Epoch 46
Validation binary_cross_entropy = 0.422303
Epoch 47
Validation binary_cross_entropy = 0.438821
Epoch 48
Validation binary_cross_entropy = 0.385833
Epoch 49
Loss = 1.5340e-03, PNorm = 89.6578, GNorm = 0.6619, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.453212
Epoch 50
Validation binary_cross_entropy = 0.472633
Epoch 51
Validation binary_cross_entropy = 0.422940
Epoch 52
Validation binary_cross_entropy = 0.452764
Epoch 53
Validation binary_cross_entropy = 0.447797
Epoch 54
Loss = 1.3357e-03, PNorm = 89.8418, GNorm = 1.2215, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.514482
Epoch 55
Validation binary_cross_entropy = 0.474822
Epoch 56
Validation binary_cross_entropy = 0.503490
Epoch 57
Validation binary_cross_entropy = 0.467965
Epoch 58
Validation binary_cross_entropy = 0.445117
Epoch 59
Loss = 4.0181e-03, PNorm = 90.0123, GNorm = 1.8247, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.445847
Epoch 60
Validation binary_cross_entropy = 0.518977
Epoch 61
Validation binary_cross_entropy = 0.515313
Epoch 62
Validation binary_cross_entropy = 0.475233
Epoch 63
Validation binary_cross_entropy = 0.463563
Epoch 64
Loss = 1.3453e-03, PNorm = 90.1726, GNorm = 1.2691, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.469209
Epoch 65
Validation binary_cross_entropy = 0.485488
Epoch 66
Validation binary_cross_entropy = 0.475635
Epoch 67
Validation binary_cross_entropy = 0.465844
Epoch 68
Validation binary_cross_entropy = 0.479869
Epoch 69
Loss = 6.7770e-04, PNorm = 90.3319, GNorm = 0.3016, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.505202
Epoch 70
Validation binary_cross_entropy = 0.514180
Epoch 71
Validation binary_cross_entropy = 0.513456
Epoch 72
Validation binary_cross_entropy = 0.503224
Epoch 73
Validation binary_cross_entropy = 0.487921
Epoch 74
Loss = 3.0676e-04, PNorm = 90.4522, GNorm = 0.5428, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.466620
Epoch 75
Validation binary_cross_entropy = 0.469024
Epoch 76
Validation binary_cross_entropy = 0.488655
Epoch 77
Validation binary_cross_entropy = 0.534620
Epoch 78
Validation binary_cross_entropy = 0.537924
Epoch 79
Loss = 7.2057e-04, PNorm = 90.5472, GNorm = 0.7620, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.486681
Epoch 80
Validation binary_cross_entropy = 0.485144
Epoch 81
Validation binary_cross_entropy = 0.543233
Epoch 82
Validation binary_cross_entropy = 0.613628
Epoch 83
Validation binary_cross_entropy = 0.578506
Epoch 84
Loss = 7.8380e-04, PNorm = 90.6608, GNorm = 1.1297, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.517150
Epoch 85
Validation binary_cross_entropy = 0.512394
Epoch 86
Validation binary_cross_entropy = 0.543612
Epoch 87
Validation binary_cross_entropy = 0.563112
Epoch 88
Validation binary_cross_entropy = 0.544111
Epoch 89
Loss = 2.2451e-03, PNorm = 90.8578, GNorm = 1.5708, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.537135
Epoch 90
Validation binary_cross_entropy = 0.500721
Epoch 91
Validation binary_cross_entropy = 0.490547
Epoch 92
Validation binary_cross_entropy = 0.474567
Epoch 93
Validation binary_cross_entropy = 0.543092
Epoch 94
Loss = 2.5201e-03, PNorm = 91.0721, GNorm = 2.2375, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.499140
Epoch 95
Validation binary_cross_entropy = 0.440164
Epoch 96
Validation binary_cross_entropy = 0.431321
Epoch 97
Validation binary_cross_entropy = 0.484452
Epoch 98
Validation binary_cross_entropy = 0.513235
Epoch 99
Loss = 2.6082e-03, PNorm = 91.2495, GNorm = 1.9936, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.512045
Epoch 100
Validation binary_cross_entropy = 0.609056
Epoch 101
Validation binary_cross_entropy = 0.616090
Epoch 102
Validation binary_cross_entropy = 0.588799
Epoch 103
Validation binary_cross_entropy = 0.621697
Epoch 104
Loss = 8.0546e-04, PNorm = 91.4120, GNorm = 1.2037, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.652911
Epoch 105
Validation binary_cross_entropy = 0.662612
Epoch 106
Validation binary_cross_entropy = 0.698493
Epoch 107
Validation binary_cross_entropy = 0.678481
Epoch 108
Validation binary_cross_entropy = 0.672700
Epoch 109
Loss = 5.2953e-04, PNorm = 91.6001, GNorm = 0.2851, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.631369
Epoch 110
Validation binary_cross_entropy = 0.622679
Epoch 111
Validation binary_cross_entropy = 0.605966
Epoch 112
Validation binary_cross_entropy = 0.631752
Epoch 113
Validation binary_cross_entropy = 0.652265
Epoch 114
Loss = 2.1562e-03, PNorm = 91.7664, GNorm = 1.2693, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.731489
Epoch 115
Validation binary_cross_entropy = 0.732048
Epoch 116
Validation binary_cross_entropy = 0.722355
Epoch 117
Validation binary_cross_entropy = 0.651227
Epoch 118
Validation binary_cross_entropy = 0.619756
Epoch 119
Loss = 1.1924e-03, PNorm = 91.9645, GNorm = 1.5542, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.599899
Epoch 120
Validation binary_cross_entropy = 0.603261
Epoch 121
Validation binary_cross_entropy = 0.611684
Epoch 122
Validation binary_cross_entropy = 0.620287
Epoch 123
Validation binary_cross_entropy = 0.634538
Epoch 124
Loss = 9.3916e-04, PNorm = 92.1130, GNorm = 0.7680, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.649498
Epoch 125
Validation binary_cross_entropy = 0.667790
Epoch 126
Validation binary_cross_entropy = 0.706351
Epoch 127
Validation binary_cross_entropy = 0.724115
Epoch 128
Validation binary_cross_entropy = 0.728928
Epoch 129
Loss = 9.8704e-04, PNorm = 92.2265, GNorm = 0.1086, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.654953
Epoch 130
Validation binary_cross_entropy = 0.650403
Epoch 131
Validation binary_cross_entropy = 0.678471
Epoch 132
Validation binary_cross_entropy = 0.802290
Epoch 133
Validation binary_cross_entropy = 0.733166
Epoch 134
Loss = 1.8789e-04, PNorm = 92.4034, GNorm = 0.2049, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.657480
Epoch 135
Validation binary_cross_entropy = 0.637822
Epoch 136
Validation binary_cross_entropy = 0.657818
Epoch 137
Validation binary_cross_entropy = 0.735063
Epoch 138
Validation binary_cross_entropy = 0.764390
Epoch 139
Loss = 1.0405e-04, PNorm = 92.5981, GNorm = 0.0987, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.722321
Epoch 140
Validation binary_cross_entropy = 0.724634
Epoch 141
Validation binary_cross_entropy = 0.805066
Epoch 142
Validation binary_cross_entropy = 0.893351
Epoch 143
Validation binary_cross_entropy = 0.785335
Epoch 144
Loss = 1.6352e-03, PNorm = 92.8081, GNorm = 3.6884, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.863882
Epoch 145
Validation binary_cross_entropy = 0.742724
Epoch 146
Validation binary_cross_entropy = 0.699902
Epoch 147
Validation binary_cross_entropy = 0.685359
Epoch 148
Validation binary_cross_entropy = 0.711849
Epoch 149
Loss = 5.9142e-04, PNorm = 93.0242, GNorm = 0.2269, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.681647
Epoch 150
Validation binary_cross_entropy = 0.670092
Epoch 151
Validation binary_cross_entropy = 0.680936
Epoch 152
Validation binary_cross_entropy = 0.677744
Epoch 153
Validation binary_cross_entropy = 0.647782
Epoch 154
Loss = 6.4351e-05, PNorm = 93.2186, GNorm = 0.1621, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.628115
Epoch 155
Validation binary_cross_entropy = 0.620459
Epoch 156
Validation binary_cross_entropy = 0.625336
Epoch 157
Validation binary_cross_entropy = 0.652992
Epoch 158
Validation binary_cross_entropy = 0.699753
Epoch 159
Loss = 2.0290e-04, PNorm = 93.3363, GNorm = 0.7972, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.748753
Epoch 160
Validation binary_cross_entropy = 0.754002
Epoch 161
Validation binary_cross_entropy = 0.731010
Epoch 162
Validation binary_cross_entropy = 0.679408
Epoch 163
Validation binary_cross_entropy = 0.674827
Epoch 164
Loss = 2.7000e-04, PNorm = 93.4461, GNorm = 1.1670, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.696298
Epoch 165
Validation binary_cross_entropy = 0.736064
Epoch 166
Validation binary_cross_entropy = 0.759859
Epoch 167
Validation binary_cross_entropy = 0.764950
Epoch 168
Validation binary_cross_entropy = 0.819716
Epoch 169
Loss = 1.4404e-03, PNorm = 93.5346, GNorm = 4.6570, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.796321
Epoch 170
Validation binary_cross_entropy = 0.732654
Epoch 171
Validation binary_cross_entropy = 0.734179
Epoch 172
Validation binary_cross_entropy = 0.744725
Epoch 173
Validation binary_cross_entropy = 0.750550
Epoch 174
Loss = 3.5927e-04, PNorm = 93.6366, GNorm = 0.0354, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.755864
Epoch 175
Validation binary_cross_entropy = 0.768670
Epoch 176
Validation binary_cross_entropy = 0.804707
Epoch 177
Validation binary_cross_entropy = 0.866093
Epoch 178
Validation binary_cross_entropy = 0.864496
Epoch 179
Loss = 6.6172e-04, PNorm = 93.7267, GNorm = 0.4684, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.785951
Epoch 180
Validation binary_cross_entropy = 0.730820
Epoch 181
Validation binary_cross_entropy = 0.722329
Epoch 182
Validation binary_cross_entropy = 0.738060
Epoch 183
Validation binary_cross_entropy = 0.782682
Epoch 184
Loss = 9.8619e-05, PNorm = 93.8114, GNorm = 0.3823, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.872487
Epoch 185
Validation binary_cross_entropy = 0.911716
Epoch 186
Validation binary_cross_entropy = 0.894868
Epoch 187
Validation binary_cross_entropy = 0.848602
Epoch 188
Validation binary_cross_entropy = 0.811153
Epoch 189
Loss = 1.7749e-04, PNorm = 93.9491, GNorm = 0.0447, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.797462
Epoch 190
Validation binary_cross_entropy = 0.805614
Epoch 191
Validation binary_cross_entropy = 0.832579
Epoch 192
Validation binary_cross_entropy = 0.865682
Epoch 193
Validation binary_cross_entropy = 0.912630
Epoch 194
Loss = 5.2204e-05, PNorm = 94.0447, GNorm = 0.1259, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.965105
Epoch 195
Validation binary_cross_entropy = 1.003485
Epoch 196
Validation binary_cross_entropy = 0.932162
Epoch 197
Validation binary_cross_entropy = 0.852009
Epoch 198
Validation binary_cross_entropy = 0.813758
Epoch 199
Loss = 9.9602e-04, PNorm = 94.1021, GNorm = 1.6980, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.800413
Epoch 200
Validation binary_cross_entropy = 0.792063
Epoch 201
Validation binary_cross_entropy = 0.790247
Epoch 202
Validation binary_cross_entropy = 0.793239
Epoch 203
Validation binary_cross_entropy = 0.795684
Epoch 204
Loss = 3.4863e-05, PNorm = 94.1479, GNorm = 0.2223, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.800466
Epoch 205
Validation binary_cross_entropy = 0.808057
Epoch 206
Validation binary_cross_entropy = 0.820482
Epoch 207
Validation binary_cross_entropy = 0.835363
Epoch 208
Validation binary_cross_entropy = 0.850985
Epoch 209
Loss = 3.6085e-05, PNorm = 94.1830, GNorm = 0.0989, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.868843
Epoch 210
Validation binary_cross_entropy = 0.896912
Epoch 211
Validation binary_cross_entropy = 0.923878
Epoch 212
Validation binary_cross_entropy = 0.946627
Epoch 213
Validation binary_cross_entropy = 0.964540
Epoch 214
Loss = 6.9699e-05, PNorm = 94.2336, GNorm = 0.0541, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.958581
Epoch 215
Validation binary_cross_entropy = 0.953493
Epoch 216
Validation binary_cross_entropy = 0.950082
Epoch 217
Validation binary_cross_entropy = 0.942317
Epoch 218
Validation binary_cross_entropy = 0.944913
Epoch 219
Loss = 4.7147e-05, PNorm = 94.2869, GNorm = 0.0539, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.957808
Epoch 220
Validation binary_cross_entropy = 0.975316
Epoch 221
Validation binary_cross_entropy = 0.952482
Epoch 222
Validation binary_cross_entropy = 0.890795
Epoch 223
Validation binary_cross_entropy = 0.871644
Epoch 224
Loss = 1.9576e-05, PNorm = 94.3490, GNorm = 0.0488, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.866920
Epoch 225
Validation binary_cross_entropy = 0.893663
Epoch 226
Validation binary_cross_entropy = 0.939474
Epoch 227
Validation binary_cross_entropy = 0.987454
Epoch 228
Validation binary_cross_entropy = 0.999055
Epoch 229
Loss = 2.9603e-05, PNorm = 94.4086, GNorm = 0.0480, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.974732
Epoch 230
Validation binary_cross_entropy = 0.938756
Epoch 231
Validation binary_cross_entropy = 0.894250
Epoch 232
Validation binary_cross_entropy = 0.882355
Epoch 233
Validation binary_cross_entropy = 0.874838
Epoch 234
Loss = 5.3515e-06, PNorm = 94.4608, GNorm = 0.0285, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.869091
Epoch 235
Validation binary_cross_entropy = 0.866721
Epoch 236
Validation binary_cross_entropy = 0.865926
Epoch 237
Validation binary_cross_entropy = 0.865338
Epoch 238
Validation binary_cross_entropy = 0.864368
Epoch 239
Loss = 1.7955e-04, PNorm = 94.5017, GNorm = 1.0792, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.855987
Epoch 240
Validation binary_cross_entropy = 0.849121
Epoch 241
Validation binary_cross_entropy = 0.845157
Epoch 242
Validation binary_cross_entropy = 0.835379
Epoch 243
Validation binary_cross_entropy = 0.830581
Epoch 244
Loss = 7.5466e-04, PNorm = 94.5299, GNorm = 0.0063, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.825000
Epoch 245
Validation binary_cross_entropy = 0.826270
Epoch 246
Validation binary_cross_entropy = 0.839350
Epoch 247
Validation binary_cross_entropy = 0.867030
Epoch 248
Validation binary_cross_entropy = 0.878583
Epoch 249
Loss = 3.2630e-04, PNorm = 94.5543, GNorm = 0.0162, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.850211
Epoch 250
Validation binary_cross_entropy = 0.844203
Epoch 251
Validation binary_cross_entropy = 0.851370
Epoch 252
Validation binary_cross_entropy = 0.863302
Epoch 253
Validation binary_cross_entropy = 0.875253
Epoch 254
Loss = 1.8492e-04, PNorm = 94.5779, GNorm = 0.0183, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.880177
Epoch 255
Validation binary_cross_entropy = 0.885601
Epoch 256
Validation binary_cross_entropy = 0.886956
Epoch 257
Validation binary_cross_entropy = 0.893081
Epoch 258
Validation binary_cross_entropy = 0.902944
Epoch 259
Loss = 1.4133e-05, PNorm = 94.6014, GNorm = 0.0335, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.913461
Epoch 260
Validation binary_cross_entropy = 0.914651
Epoch 261
Validation binary_cross_entropy = 0.911617
Epoch 262
Validation binary_cross_entropy = 0.910830
Epoch 263
Validation binary_cross_entropy = 0.911159
Epoch 264
Loss = 2.0282e-04, PNorm = 94.6292, GNorm = 0.7171, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 0.910773
Epoch 265
Validation binary_cross_entropy = 0.910107
Epoch 266
Validation binary_cross_entropy = 0.910721
Epoch 267
Validation binary_cross_entropy = 0.912819
Epoch 268
Validation binary_cross_entropy = 0.915914
Epoch 269
Loss = 5.7800e-05, PNorm = 94.6498, GNorm = 0.3642, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.918839
Epoch 270
Validation binary_cross_entropy = 0.921338
Epoch 271
Validation binary_cross_entropy = 0.925487
Epoch 272
Validation binary_cross_entropy = 0.929612
Epoch 273
Validation binary_cross_entropy = 0.933696
Epoch 274
Loss = 1.6218e-06, PNorm = 94.6629, GNorm = 0.0019, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 0.937201
Epoch 275
Validation binary_cross_entropy = 0.940012
Epoch 276
Validation binary_cross_entropy = 0.942341
Epoch 277
Validation binary_cross_entropy = 0.944100
Epoch 278
Validation binary_cross_entropy = 0.945465
Epoch 279
Loss = 2.5539e-05, PNorm = 94.6698, GNorm = 0.1097, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.946752
Epoch 280
Validation binary_cross_entropy = 0.949556
Epoch 281
Validation binary_cross_entropy = 0.953133
Epoch 282
Validation binary_cross_entropy = 0.955806
Epoch 283
Validation binary_cross_entropy = 0.958084
Epoch 284
Loss = 6.7425e-06, PNorm = 94.6765, GNorm = 0.0017, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 0.959812
Epoch 285
Validation binary_cross_entropy = 0.961533
Epoch 286
Validation binary_cross_entropy = 0.963684
Epoch 287
Validation binary_cross_entropy = 0.965717
Epoch 288
Validation binary_cross_entropy = 0.966429
Epoch 289
Loss = 3.3105e-06, PNorm = 94.6811, GNorm = 0.0079, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 0.966705
Epoch 290
Validation binary_cross_entropy = 0.977154
Epoch 291
Validation binary_cross_entropy = 0.996689
Epoch 292
Validation binary_cross_entropy = 1.014812
Epoch 293
Validation binary_cross_entropy = 1.016946
Epoch 294
Loss = 1.2444e-04, PNorm = 94.6930, GNorm = 1.4337, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.993681
Epoch 295
Validation binary_cross_entropy = 0.961612
Epoch 296
Validation binary_cross_entropy = 0.941604
Epoch 297
Validation binary_cross_entropy = 0.929677
Epoch 298
Validation binary_cross_entropy = 0.922948
Epoch 299
Loss = 2.9012e-06, PNorm = 94.7293, GNorm = 0.0172, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.918609
Model 0 best validation binary_cross_entropy = 0.278535 on epoch 0
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.244064
Ensemble test binary_cross_entropy = 0.244064
Fold 9
Command line
python /home/saxelrod/Repo/projects/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 6,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/train_binary_cross_entropy.npz'],
 'features_scaling': False,
 'features_size': 300,
 'ffn_hidden_size': 1800,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'hidden_size': 1800,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'minimize_score': True,
 'multiclass_num_classes': 3,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/fps_from_att_k1_yes_prob_feats_mpnn_from_binary_cross_entropy/fold_9',
 'save_smiles_splits': False,
 'seed': 9,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/test_binary_cross_entropy.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/fps_from_att_k1_yes_prob/val_binary_cross_entropy.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_column': None,
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': 485,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): MPNEncoder(
      (dropout_layer): Dropout(p=0.1, inplace=False)
      (act_func): ReLU()
      (W_i): Linear(in_features=147, out_features=1800, bias=False)
      (W_h): Linear(in_features=1800, out_features=1800, bias=False)
      (W_o): Linear(in_features=1933, out_features=1800, bias=True)
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=2100, out_features=1800, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=1800, out_features=1800, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=1800, out_features=1, bias=True)
  )
)
Number of parameters = 14,011,201
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.306504
Best score so far is 0.306504 at epoch 0 
Epoch 1
Validation binary_cross_entropy = 0.539474
Epoch 2
Validation binary_cross_entropy = 0.342378
Epoch 3
Validation binary_cross_entropy = 0.575648
Epoch 4
Loss = 1.0643e-02, PNorm = 87.7243, GNorm = 2.7026, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.288970
Best score so far is 0.288970 at epoch 4 
Epoch 5
Validation binary_cross_entropy = 0.575234
Epoch 6
Validation binary_cross_entropy = 0.340120
Epoch 7
Validation binary_cross_entropy = 0.365888
Epoch 8
Validation binary_cross_entropy = 0.450985
Epoch 9
Loss = 6.2361e-03, PNorm = 88.0295, GNorm = 1.5641, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.409813
Epoch 10
Validation binary_cross_entropy = 0.371646
Epoch 11
Validation binary_cross_entropy = 0.331867
Epoch 12
Validation binary_cross_entropy = 0.413737
Epoch 13
Validation binary_cross_entropy = 0.331109
Epoch 14
Loss = 3.3709e-03, PNorm = 88.3713, GNorm = 1.9710, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.351794
Epoch 15
Validation binary_cross_entropy = 0.346934
Epoch 16
Validation binary_cross_entropy = 0.378323
Epoch 17
Validation binary_cross_entropy = 0.388053
Epoch 18
Validation binary_cross_entropy = 0.345561
Epoch 19
Loss = 2.1174e-03, PNorm = 88.6505, GNorm = 2.0950, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.463564
Epoch 20
Validation binary_cross_entropy = 0.462560
Epoch 21
Validation binary_cross_entropy = 0.403640
Epoch 22
Validation binary_cross_entropy = 0.591540
Epoch 23
Validation binary_cross_entropy = 0.360378
Epoch 24
Loss = 5.8591e-03, PNorm = 88.8773, GNorm = 4.0945, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.336536
Epoch 25
Validation binary_cross_entropy = 0.462226
Epoch 26
Validation binary_cross_entropy = 0.383290
Epoch 27
Validation binary_cross_entropy = 0.381899
Epoch 28
Validation binary_cross_entropy = 0.373105
Epoch 29
Loss = 2.3832e-03, PNorm = 89.0693, GNorm = 1.4440, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.464930
Epoch 30
Validation binary_cross_entropy = 0.454794
Epoch 31
Validation binary_cross_entropy = 0.385636
Epoch 32
Validation binary_cross_entropy = 0.364585
Epoch 33
Validation binary_cross_entropy = 0.390658
Epoch 34
Loss = 2.8784e-03, PNorm = 89.2354, GNorm = 2.5507, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.400656
Epoch 35
Validation binary_cross_entropy = 0.392373
Epoch 36
Validation binary_cross_entropy = 0.380052
Epoch 37
Validation binary_cross_entropy = 0.359527
Epoch 38
Validation binary_cross_entropy = 0.366369
Epoch 39
Loss = 2.5489e-03, PNorm = 89.4172, GNorm = 1.3439, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.378145
Epoch 40
Validation binary_cross_entropy = 0.393220
Epoch 41
Validation binary_cross_entropy = 0.424849
Epoch 42
Validation binary_cross_entropy = 0.413589
Epoch 43
Validation binary_cross_entropy = 0.469299
Epoch 44
Loss = 2.5574e-03, PNorm = 89.5972, GNorm = 1.6708, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.455421
Epoch 45
Validation binary_cross_entropy = 0.483183
Epoch 46
Validation binary_cross_entropy = 0.492554
Epoch 47
Validation binary_cross_entropy = 0.491338
Epoch 48
Validation binary_cross_entropy = 0.508610
Epoch 49
Loss = 2.2154e-03, PNorm = 89.7712, GNorm = 0.9425, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.512364
Epoch 50
Validation binary_cross_entropy = 0.513822
Epoch 51
Validation binary_cross_entropy = 0.520530
Epoch 52
Validation binary_cross_entropy = 0.535796
Epoch 53
Validation binary_cross_entropy = 0.507710
Epoch 54
Loss = 5.8933e-04, PNorm = 89.9101, GNorm = 1.4744, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.498401
Epoch 55
Validation binary_cross_entropy = 0.526463
Epoch 56
Validation binary_cross_entropy = 0.536900
Epoch 57
Validation binary_cross_entropy = 0.550388
Epoch 58
Validation binary_cross_entropy = 0.544977
Epoch 59
Loss = 4.6442e-04, PNorm = 90.0291, GNorm = 0.3708, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.533662
Epoch 60
Validation binary_cross_entropy = 0.539024
Epoch 61
Validation binary_cross_entropy = 0.517772
Epoch 62
Validation binary_cross_entropy = 0.543046
Epoch 63
Validation binary_cross_entropy = 0.657359
Epoch 64
Loss = 3.7522e-03, PNorm = 90.1611, GNorm = 1.2387, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.537310
Epoch 65
Validation binary_cross_entropy = 0.529554
Epoch 66
Validation binary_cross_entropy = 0.565222
Epoch 67
Validation binary_cross_entropy = 0.586167
Epoch 68
Validation binary_cross_entropy = 0.598324
Epoch 69
Loss = 9.9635e-04, PNorm = 90.3334, GNorm = 1.2591, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.585426
Epoch 70
Validation binary_cross_entropy = 0.586056
Epoch 71
Validation binary_cross_entropy = 0.587788
Epoch 72
Validation binary_cross_entropy = 0.547711
Epoch 73
Validation binary_cross_entropy = 0.587784
Epoch 74
Loss = 2.5172e-04, PNorm = 90.5122, GNorm = 0.1542, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.612506
Epoch 75
Validation binary_cross_entropy = 0.633525
Epoch 76
Validation binary_cross_entropy = 0.557835
Epoch 77
Validation binary_cross_entropy = 0.589887
Epoch 78
Validation binary_cross_entropy = 0.554048
Epoch 79
Loss = 2.3634e-03, PNorm = 90.6818, GNorm = 2.4428, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.555829
Epoch 80
Validation binary_cross_entropy = 0.588637
Epoch 81
Validation binary_cross_entropy = 0.647511
Epoch 82
Validation binary_cross_entropy = 0.583850
Epoch 83
Validation binary_cross_entropy = 0.571604
Epoch 84
Loss = 1.4682e-03, PNorm = 90.8777, GNorm = 1.0128, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.564910
Epoch 85
Validation binary_cross_entropy = 0.518812
Epoch 86
Validation binary_cross_entropy = 0.514852
Epoch 87
Validation binary_cross_entropy = 0.599816
Epoch 88
Validation binary_cross_entropy = 0.597361
Epoch 89
Loss = 8.0036e-04, PNorm = 91.0642, GNorm = 1.7943, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.591086
Epoch 90
Validation binary_cross_entropy = 0.591866
Epoch 91
Validation binary_cross_entropy = 0.632742
Epoch 92
Validation binary_cross_entropy = 0.664958
Epoch 93
Validation binary_cross_entropy = 0.624217
Epoch 94
Loss = 4.6548e-04, PNorm = 91.2435, GNorm = 0.7000, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.603855
Epoch 95
Validation binary_cross_entropy = 0.576725
Epoch 96
Validation binary_cross_entropy = 0.569323
Epoch 97
Validation binary_cross_entropy = 0.578513
Epoch 98
Validation binary_cross_entropy = 0.592202
Epoch 99
Loss = 1.7706e-03, PNorm = 91.3989, GNorm = 1.1298, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.609435
Epoch 100
Validation binary_cross_entropy = 0.638056
Epoch 101
Validation binary_cross_entropy = 0.620762
Epoch 102
Validation binary_cross_entropy = 0.593302
Epoch 103
Validation binary_cross_entropy = 0.594104
Epoch 104
Loss = 5.1538e-04, PNorm = 91.5254, GNorm = 0.3641, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.618711
Epoch 105
Validation binary_cross_entropy = 0.597255
Epoch 106
Validation binary_cross_entropy = 0.586357
Epoch 107
Validation binary_cross_entropy = 0.590438
Epoch 108
Validation binary_cross_entropy = 0.605026
Epoch 109
Loss = 3.3815e-04, PNorm = 91.6340, GNorm = 0.8047, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.624203
Epoch 110
Validation binary_cross_entropy = 0.644431
Epoch 111
Validation binary_cross_entropy = 0.670162
Epoch 112
Validation binary_cross_entropy = 0.662809
Epoch 113
Validation binary_cross_entropy = 0.654863
Epoch 114
Loss = 5.2529e-04, PNorm = 91.7189, GNorm = 2.0664, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.667170
Epoch 115
Validation binary_cross_entropy = 0.701210
Epoch 116
Validation binary_cross_entropy = 0.761464
Epoch 117
Validation binary_cross_entropy = 0.721120
Epoch 118
Validation binary_cross_entropy = 0.766323
Epoch 119
Loss = 2.0342e-03, PNorm = 91.8583, GNorm = 3.3403, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.732282
Epoch 120
Validation binary_cross_entropy = 0.737153
Epoch 121
Validation binary_cross_entropy = 0.877020
Epoch 122
Validation binary_cross_entropy = 0.662413
Epoch 123
Validation binary_cross_entropy = 0.664160
Epoch 124
Loss = 2.2214e-03, PNorm = 92.1058, GNorm = 2.9455, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.670118
Epoch 125
Validation binary_cross_entropy = 0.739468
Epoch 126
Validation binary_cross_entropy = 0.625157
Epoch 127
Validation binary_cross_entropy = 0.602175
Epoch 128
Validation binary_cross_entropy = 0.569692
Epoch 129
Loss = 4.4493e-04, PNorm = 92.3569, GNorm = 0.6206, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.517848
Epoch 130
Validation binary_cross_entropy = 0.503300
Epoch 131
Validation binary_cross_entropy = 0.529427
Epoch 132
Validation binary_cross_entropy = 0.525881
Epoch 133
Validation binary_cross_entropy = 0.488017
Epoch 134
Loss = 1.0773e-03, PNorm = 92.5901, GNorm = 1.3311, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.493294
Epoch 135
Validation binary_cross_entropy = 0.492501
Epoch 136
Validation binary_cross_entropy = 0.538620
Epoch 137
Validation binary_cross_entropy = 0.562936
Epoch 138
Validation binary_cross_entropy = 0.555590
Epoch 139
Loss = 1.5711e-03, PNorm = 92.7875, GNorm = 0.5722, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.523931
Epoch 140
Validation binary_cross_entropy = 0.526421
Epoch 141
Validation binary_cross_entropy = 0.493024
Epoch 142
Validation binary_cross_entropy = 0.521846
Epoch 143
Validation binary_cross_entropy = 0.595964
Epoch 144
Loss = 7.5115e-04, PNorm = 92.9576, GNorm = 0.9661, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.566173
Epoch 145
Validation binary_cross_entropy = 0.544681
Epoch 146
Validation binary_cross_entropy = 0.556737
Epoch 147
Validation binary_cross_entropy = 0.583344
Epoch 148
Validation binary_cross_entropy = 0.597113
Epoch 149
Loss = 8.7576e-04, PNorm = 93.0703, GNorm = 0.1645, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.609575
Epoch 150
Validation binary_cross_entropy = 0.625143
Epoch 151
Validation binary_cross_entropy = 0.609223
Epoch 152
Validation binary_cross_entropy = 0.587431
Epoch 153
Validation binary_cross_entropy = 0.574814
Epoch 154
Loss = 2.7187e-04, PNorm = 93.1644, GNorm = 0.4897, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.572065
Epoch 155
Validation binary_cross_entropy = 0.574802
Epoch 156
Validation binary_cross_entropy = 0.572854
Epoch 157
Validation binary_cross_entropy = 0.640507
Epoch 158
Validation binary_cross_entropy = 0.757619
Epoch 159
Loss = 9.2253e-04, PNorm = 93.2797, GNorm = 2.1017, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.702362
Epoch 160
Validation binary_cross_entropy = 0.611588
Epoch 161
Validation binary_cross_entropy = 0.562863
Epoch 162
Validation binary_cross_entropy = 0.550041
Epoch 163
Validation binary_cross_entropy = 0.551043
Epoch 164
Loss = 4.3163e-04, PNorm = 93.3944, GNorm = 0.0249, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.593308
Epoch 165
Validation binary_cross_entropy = 0.640812
Epoch 166
Validation binary_cross_entropy = 0.680419
Epoch 167
Validation binary_cross_entropy = 0.621096
Epoch 168
Validation binary_cross_entropy = 0.583639
Epoch 169
Loss = 7.5577e-04, PNorm = 93.4947, GNorm = 1.4057, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 0.563137
Epoch 170
Validation binary_cross_entropy = 0.558142
Epoch 171
Validation binary_cross_entropy = 0.567052
Epoch 172
Validation binary_cross_entropy = 0.608473
Epoch 173
Validation binary_cross_entropy = 0.644957
Epoch 174
Loss = 9.3224e-04, PNorm = 93.5887, GNorm = 1.0275, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.595391
Epoch 175
Validation binary_cross_entropy = 0.536033
Epoch 176
Validation binary_cross_entropy = 0.550121
Epoch 177
Validation binary_cross_entropy = 0.561892
Epoch 178
Validation binary_cross_entropy = 0.561301
Epoch 179
Loss = 1.1187e-03, PNorm = 93.6757, GNorm = 1.9034, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.568965
Epoch 180
Validation binary_cross_entropy = 0.598954
Epoch 181
Validation binary_cross_entropy = 0.635021
Epoch 182
Validation binary_cross_entropy = 0.631921
Epoch 183
Validation binary_cross_entropy = 0.595273
Epoch 184
Loss = 6.3673e-04, PNorm = 93.7740, GNorm = 1.7632, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.584243
Epoch 185
Validation binary_cross_entropy = 0.597503
Epoch 186
Validation binary_cross_entropy = 0.612711
Epoch 187
Validation binary_cross_entropy = 0.612303
Epoch 188
Validation binary_cross_entropy = 0.606047
Epoch 189
Loss = 7.3046e-05, PNorm = 93.8714, GNorm = 0.0743, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.606152
Epoch 190
Validation binary_cross_entropy = 0.606935
Epoch 191
Validation binary_cross_entropy = 0.613561
Epoch 192
Validation binary_cross_entropy = 0.621466
Epoch 193
Validation binary_cross_entropy = 0.629579
Epoch 194
Loss = 2.4697e-05, PNorm = 93.9360, GNorm = 0.1196, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.638211
Epoch 195
Validation binary_cross_entropy = 0.657323
Epoch 196
Validation binary_cross_entropy = 0.690136
Epoch 197
Validation binary_cross_entropy = 0.720939
Epoch 198
Validation binary_cross_entropy = 0.737491
Epoch 199
Loss = 4.0410e-05, PNorm = 93.9874, GNorm = 0.2237, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.745845
Epoch 200
Validation binary_cross_entropy = 0.750523
Epoch 201
Validation binary_cross_entropy = 0.727954
Epoch 202
Validation binary_cross_entropy = 0.694796
Epoch 203
Validation binary_cross_entropy = 0.673310
Epoch 204
Loss = 1.8567e-04, PNorm = 94.0263, GNorm = 0.0178, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 0.671249
Epoch 205
Validation binary_cross_entropy = 0.675097
Epoch 206
Validation binary_cross_entropy = 0.673279
Epoch 207
Validation binary_cross_entropy = 0.686056
Epoch 208
Validation binary_cross_entropy = 0.714729
Epoch 209
Loss = 3.8102e-05, PNorm = 94.0886, GNorm = 0.0529, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 0.740739
Epoch 210
Validation binary_cross_entropy = 0.767178
Epoch 211
Validation binary_cross_entropy = 0.821259
Epoch 212
Validation binary_cross_entropy = 0.895881
Epoch 213
Validation binary_cross_entropy = 0.882626
Epoch 214
Loss = 8.9413e-04, PNorm = 94.1798, GNorm = 0.0197, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 0.761422
Epoch 215
Validation binary_cross_entropy = 0.748239
Epoch 216
Validation binary_cross_entropy = 0.742450
Epoch 217
Validation binary_cross_entropy = 0.763088
Epoch 218
Validation binary_cross_entropy = 0.761967
Epoch 219
Loss = 1.3434e-03, PNorm = 94.2873, GNorm = 0.4977, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 0.712450
Epoch 220
Validation binary_cross_entropy = 0.687549
Epoch 221
Validation binary_cross_entropy = 0.680915
Epoch 222
Validation binary_cross_entropy = 0.688555
Epoch 223
Validation binary_cross_entropy = 0.721697
Epoch 224
Loss = 1.8365e-04, PNorm = 94.3911, GNorm = 0.0467, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 0.758132
Epoch 225
Validation binary_cross_entropy = 0.785906
Epoch 226
Validation binary_cross_entropy = 0.807078
Epoch 227
Validation binary_cross_entropy = 0.817437
Epoch 228
Validation binary_cross_entropy = 0.808146
Epoch 229
Loss = 1.1358e-04, PNorm = 94.4917, GNorm = 0.3128, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.799479
Epoch 230
Validation binary_cross_entropy = 0.794555
Epoch 231
Validation binary_cross_entropy = 0.797973
Epoch 232
Validation binary_cross_entropy = 0.808512
Epoch 233
Validation binary_cross_entropy = 0.819683
Epoch 234
Loss = 6.4346e-05, PNorm = 94.5693, GNorm = 0.2618, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 0.830589
Epoch 235
Validation binary_cross_entropy = 0.841356
Epoch 236
Validation binary_cross_entropy = 0.851342
Epoch 237
Validation binary_cross_entropy = 0.860312
Epoch 238
Validation binary_cross_entropy = 0.866924
Epoch 239
Loss = 5.0767e-05, PNorm = 94.6222, GNorm = 0.2676, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 0.871853
Epoch 240
Validation binary_cross_entropy = 0.873499
Epoch 241
Validation binary_cross_entropy = 0.880124
Epoch 242
Validation binary_cross_entropy = 0.884764
Epoch 243
Validation binary_cross_entropy = 0.887962
Epoch 244
Loss = 2.0859e-04, PNorm = 94.6522, GNorm = 0.7786, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 0.895006
Epoch 245
Validation binary_cross_entropy = 0.907099
Epoch 246
Validation binary_cross_entropy = 0.917544
Epoch 247
Validation binary_cross_entropy = 0.918908
Epoch 248
Validation binary_cross_entropy = 0.915158
Epoch 249
Loss = 8.1642e-05, PNorm = 94.6805, GNorm = 0.2747, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 0.901592
Epoch 250
Validation binary_cross_entropy = 0.888372
Epoch 251
Validation binary_cross_entropy = 0.878626
Epoch 252
Validation binary_cross_entropy = 0.873494
Epoch 253
Validation binary_cross_entropy = 0.871931
Epoch 254
Loss = 1.1114e-04, PNorm = 94.7071, GNorm = 0.0303, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 0.878259
Epoch 255
Validation binary_cross_entropy = 0.895198
Epoch 256
Validation binary_cross_entropy = 0.918201
Epoch 257
Validation binary_cross_entropy = 0.935311
Epoch 258
Validation binary_cross_entropy = 0.934631
Epoch 259
Loss = 6.1094e-05, PNorm = 94.7469, GNorm = 0.0022, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 0.900501
Epoch 260
Validation binary_cross_entropy = 0.871867
Epoch 261
Validation binary_cross_entropy = 0.890074
Epoch 262
Validation binary_cross_entropy = 0.928657
Epoch 263
Validation binary_cross_entropy = 1.002189
Epoch 264
Loss = 3.4002e-05, PNorm = 94.8202, GNorm = 0.1714, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.057226
Epoch 265
Validation binary_cross_entropy = 1.073489
Epoch 266
Validation binary_cross_entropy = 1.054840
Epoch 267
Validation binary_cross_entropy = 1.009031
Epoch 268
Validation binary_cross_entropy = 0.973667
Epoch 269
Loss = 5.3602e-04, PNorm = 94.9021, GNorm = 1.7055, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 0.964498
Epoch 270
Validation binary_cross_entropy = 0.968491
Epoch 271
Validation binary_cross_entropy = 0.996641
Epoch 272
Validation binary_cross_entropy = 1.025842
Epoch 273
Validation binary_cross_entropy = 1.057575
Epoch 274
Loss = 1.6937e-04, PNorm = 94.9627, GNorm = 1.2500, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.050985
Epoch 275
Validation binary_cross_entropy = 1.012348
Epoch 276
Validation binary_cross_entropy = 0.982381
Epoch 277
Validation binary_cross_entropy = 0.962602
Epoch 278
Validation binary_cross_entropy = 0.949470
Epoch 279
Loss = 2.7909e-05, PNorm = 95.0219, GNorm = 0.2482, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 0.939142
Epoch 280
Validation binary_cross_entropy = 0.932232
Epoch 281
Validation binary_cross_entropy = 0.928727
Epoch 282
Validation binary_cross_entropy = 0.925671
Epoch 283
Validation binary_cross_entropy = 0.929023
Epoch 284
Loss = 5.6858e-04, PNorm = 95.0649, GNorm = 0.0064, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.041642
Epoch 285
Validation binary_cross_entropy = 1.190257
Epoch 286
Validation binary_cross_entropy = 1.263018
Epoch 287
Validation binary_cross_entropy = 1.223845
Epoch 288
Validation binary_cross_entropy = 1.123472
Epoch 289
Loss = 9.1147e-06, PNorm = 95.2126, GNorm = 0.0370, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.074898
Epoch 290
Validation binary_cross_entropy = 1.052827
Epoch 291
Validation binary_cross_entropy = 1.035720
Epoch 292
Validation binary_cross_entropy = 1.014275
Epoch 293
Validation binary_cross_entropy = 0.983454
Epoch 294
Loss = 1.2308e-05, PNorm = 95.3363, GNorm = 0.0311, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 0.976804
Epoch 295
Validation binary_cross_entropy = 0.986063
Epoch 296
Validation binary_cross_entropy = 0.957007
Epoch 297
Validation binary_cross_entropy = 0.941832
Epoch 298
Validation binary_cross_entropy = 0.936390
Epoch 299
Loss = 5.0701e-05, PNorm = 95.4191, GNorm = 0.0534, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 0.938038
Model 0 best validation binary_cross_entropy = 0.288970 on epoch 4
Loading pretrained parameter "encoder.encoder.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.W_i.weight".
Loading pretrained parameter "encoder.encoder.W_h.weight".
Loading pretrained parameter "encoder.encoder.W_o.weight".
Loading pretrained parameter "encoder.encoder.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.208086
Ensemble test binary_cross_entropy = 0.208086
10-fold cross validation
Seed 0 ==> test binary_cross_entropy = 0.220527
Seed 1 ==> test binary_cross_entropy = 0.240793
Seed 2 ==> test binary_cross_entropy = 0.149826
Seed 3 ==> test binary_cross_entropy = 0.245410
Seed 4 ==> test binary_cross_entropy = 0.254593
Seed 5 ==> test binary_cross_entropy = 0.264874
Seed 6 ==> test binary_cross_entropy = 0.182453
Seed 7 ==> test binary_cross_entropy = 0.164410
Seed 8 ==> test binary_cross_entropy = 0.244064
Seed 9 ==> test binary_cross_entropy = 0.208086
Overall test binary_cross_entropy = 0.217503 +/- 0.037910
